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1 . INTRODUCTION 


Currently  the  U.  S.  Anny  Is  developing  a system  for  the  acquisition, 
tracking,  and  laser  designation  of  military  targets  fn»!n  helicopters. 

The  Anny  Is  also  developing  terminal  homing  guided  weapons  capable  of 
destroying  designated  military  targets.  The  Amy  is  also  beginning  to 
Investigate  missiles  which  have  Imaging  seekers  rather  than  laser  seekers 
to  provide  guidance.  This  provides  a true  fire  and  forget  weapor-  ystem. 
Very  little  attention  has  been  given  to-date  to  the  problem  of  hand-off 
of  a target  from  the  acquisition  system  to  the  Imaging  missile  seeker. 

This  task  can  be  accomplished  by  the  gunner  using  the  video  display  from 
both  systems  on  his  monitor.  However,  this  technique  1s  time  consuming 
resulting  In  increased  helicopter  exposure  time.  As  a result  the  U.  S. 
Amy  Missile  Corrmand,  Huntsville,  Alabama  let  a contract  with  the 
Engineering  Experiment  Station,  Auburn  University  to  study  and  make  a 
recommendation  for  a system  to  accomplish  automatic  hand-off  of  targets 
from  designators  to  imaging  missile  seekers  using  correlation  techniques.  , 
This  report  presents  the  result  of  that  sf^^ort. 

The  overall  system  coiisists  of  the  helicopter  with  pilot  and 
gunner  displays  and  controls,  the  pointing  and  tracking  system  (PIS), 
either  integrally  mounted  or  on  the  stores  wing  of  the  helicopter  as 
shown  in  Figure  1,  and  the  missiles  located  in  a stores  rack  also 
mounted  on  the  wing  of  the  helicopter.  The  helicopter  supplies  power 
and  other  electronic  support  hardware  for  both  systems.  The  pointing 
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and  tracking  system  typically  consists  of  an  optics  train,  11ne>of>s1ght 
(LOS)  stabilization  system,  day  TV  and  forward  looking  Infra-red  (FLIR) 
Imaging  systems,  manual  and  autotrack  system,  laser  range  finder,  and 
associated  electronics  section.  The  Imaging  missile  seeker  could  be 
a day  TV  system  or  a FLIR.  The  current  advanced  television  seeker 
(ATVS)  being  tested  by  the  Amy  [Ij  Is  composed  of  optical,  day  tele- 
vision camera,  TV  tracker,  mode  control,  stabilization  r.nd  power  supply 
subsystems. 


Figure  1.  PTS  equipment  location  on  AH-1  helicopter. 
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In  order  to  accomplish  hand-off  using  correlation  techniques, 
the  two  signals  being  correlated  must  be  similar.  If  both  the  PTS 
and  missile  seeker  are  day  TV  systems  sensitive  In  the  same  rela- 
tive optical  spectrum,  correlation  can  be  accomplished  with  very 
little  signal  preprocessing.  However,  If  the  two  signals  are 
dissimlllar  such  as  from  a day  TV  and  FLIR,  then  correlation  possibly 
can  be  accomplished  but  only  after  considerable  preprocessing  of  the 
signals.  For  example,  correlation  on  edges  alone  might  work  In  this 
case.  The  correlator  developed  In  this  report  Is  for  two  day  - 
TV  sensors.  The  problem  of  dissimilar  sensors  is  discussed,  however. 
In  Chapter  III. 


A.  Scope  of  Work 

The  scope  of  work  performed  under  this  contract  was  to  analyze  and 
make  recommendations  on  the  hardware  and  software  required  to  be  used 
in  "handing-off"  a target  from  a target  acquisition  device  to  an  imaging 
missile  seeker.  The  function  of  an  automatic  correlator  is  to  compare 
the  video  signals  from  the  high  resolution  pointing  and  tracking  system 
(PTS)  and  from  the  missile  shaker  and  to  generate  error  signals  which 
can  be  used  to  drive  the  missiie  seeker  such  that  its  aimpoint  coincides 
with  that  of  the  PTS.  The  scope  of  work  as  written  in  the  contract 
is  given  in  the  following  six  paragraphs  for  completeness. 


l 

I 


The  first  phase  will  be  to  choose  one  or  more  candidate  correlation 
approaches.  This  will  be  done  by  surveying  the  literature  for  information 
on  existing  correlation  techniques.  It  is  realized  that  some  difficulty 
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Mty  arlst  since  nost  conpanles  have  developed  correlation  techniques 
using  their  onn  funds  and  therefore  consider  them  proprietary.  It 
Is  likely,  however,  that  some  Information  on  general  correlation 
tedmiques  Is  available  In  the  open  literature. 

Along  with  such  Information  as  Is  available  on  existing  techniques, 
an  Independent  solution  to  the  "hand-off  problem  will  be  proposed 
considering  the  variables  Involved  (I.e.,  flelds-of  view,  system 
resolutions,  stabilization,  and  others).  The  approach  will  be 
to  consider  first  the  most  accurate  and  complex  correlation  technique 
and  propose  trade-offs  In  Phase  Three. 

Since  the  target  may  not  necessarily  be  In  the  seeker  field  of 
view  (FOV),  an  algorithm  will  be  proposed  to  slew  the  seeker  TV  until 
the  target  Is  In  Its  FOV.  Of  course,  it  Is  expected  that  this  operation 
will  be  very  time  consuming  and  should  be  avoided  If  possible.  One 
possible  alternative  Is  to  determine  and  store  In  the  fire  control 
computer  boresight  errors  while  In  flight  before  a target  Is  engaged. 

The  trade-off  of  additional  software  and  computer  memory  required  by 
these  two  methods  will  be  considered. 

Phase  Two,  which  will  actually  be  conducted  concurrently  with 
Phase  One,  will  consist  of  an  Investigation  of  the  hardware  and  software 
needed  for  various  combinations  of  PTS  and  missile  seeker  configurations. 
Consideration  will  be  given  to  differing  flelds-of-view,  scan  rates, 
stabilization  of  TV  cameras,  boresight,  and  TV  format  (such  as  aspect 
ratio,  frames  per  second,  etc.).  The  contractor  will  consider  various 
combinations  and  recommend  the  hardware  and  software  necessary  to 
'implement  each. 
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Phase  Three  will  consist  of  the  actual  analysis  of  complexity 
of  the  hardware/software  necessary  to  Implement  the  various  combinations 
of  Phase  Two.  This  analysis  will  be  presented  In  such  a way  that 
MICOM  will  be  able  to  use  It  to  make  a trade-off  of  the  various 
options  available  (e.g.,  complex/ versatile  vs.  slmple/closely  bounded). 

In  Phase  Four  a correlation  technique,  along  with  the  system 
constraints,  will  be  chosen  in  concert  with  the  MICOM  technical 
director.  This  decision  will  be  based  on  the  results  of  the  first 
three  phases.  The  hardware  and  software  to  implement  this  approach 
will  be  completely  specified,  and  the  correlation  algorithm  will 
be  developed. 


B,  Organization  of  Report 

All  of  the  work  outlined  in  the  Scope  of  Work  has  been  completed 
and  1*?  documented  in  this  final  report.  Chapter  two  formulates  the 
hand-off  problem  in  greater  detail  and  gives  the  various  correlation 
techniques  considered.  As  expected,  all  companies  contacted  either 
did  not  respond  or  gave  very  little  detail  into  their  correlation 
algorithms.  As  a result,  the  effort  on  this  contract  started  with 
a literature  search  of  correlation  techniques,  defining  the  most 
accurate  but  most  time  consuming  methods,  and  making  trade-offs  in 
order  to  arrive  at  an  implementable  solution. 

Chapter  three  goes  into  detail  on  the  many  correlation  system 
considerations  which  had  to  be  resolved  before  an  implementable  solution 
could  be  attained.  These  considerations  range  from  the  system  optics. 


14 

TV  sensor  and  TV  format,  electronic  preprocessing,  sampling  rates, 
tMO  system  resolutions,  vibration  analysis,  slaving  of  two  systems 
before  correlation,  derivation  of  error  signals  to  drive  the  seeker 
gimbals,  and  others. 

Two  possible  Implementations  of  the  correlation  algorithm  are 
given  In  Chapter  four.  The  first  Implementation  was  considered  to  be 
the  state-of-the-art  up  until  October  when  TRW  publicly  announed  their 
64  bit,  20  MHz  bipolar  correlator  LSI  circuit.  The  second  implementation 
Incorporates  this  TRW  chip  and  significantly  sln^lifles  the 
hardware.  The  reason  for  giving  both  Implementations  is  to  emphasize 
how  LSI  state-of-the-art  can  effect  the  real-time  correlation  problem. 

It  is  known  that  TRW,  and  possibly  other  companies,  is  working  on 
another  LSI  correlator  chip  which  will  simplify  the  implementation 
further  or  make  possible  an  implementation  which  now  is  not  feasible 
for  real-time  correlation. 

Chapter  five  gives  the  conclusions  and  recomnendatlons  resulting 
from  the  above  work.  Questions  which  arose  during  the  course  of  this 
work  but  were  not  answered  due  to  lack  of  time  or  lack  of  simulation 
facilities  are  also  given. 


2.  FORMULATION  AND  COMPARISON  OF  CORRELATION  ALGORITHMS 


In  this  chapter  the  basic  correlation  problem  is  formulated,  four 
methods  are  presented  to  solve  the  problem,  and  finally  a comparison 
of  the  four  methods  is  given.  Two  of  the  methods  are  shown  to  be 
impractical  at  the  present  state-of-the-art  because  of  computational 
burden. 


A.  Geometric  Layout  of  PTS  and  Missile  Seeker  TV  Images 

For  the  purpose  of  designing  a correlator,  this  report  will 
consider  hypothetical  PTS  and  missile  seeker  day  TV  systems.  To 
design  a correlator  for  a particular  PTS  and  missile  seeker,  some  of 
the  design  parameters  will  have  to  be  changed  but  the  basic  procedure 
will  be  identical.  Consider  a PTS  system  with  a vertical  narrow  field 
of  view  (NFOV)  of  .5°.  a 525  line  standard  TV  system  with  2:1  interlace, 

60  Hz  field  rate,  or  30  Hz  frame  rate,  and  a 1:1  aspect  ratio.  The 
sensor  TV  will  be  assumed  to  have  a 2®  vertical  FOV  with  4:3  aspect  ratio, 
a 525  line  standard  TV  system  with  2:1  interlace  60  Hz  field  rate, 
or  30  Hz  frame  rate.  Both  systems  because  of  vertical  retrace  will 
be  considered  to  have  480  active  TV  lines. 

One  of  the  fundamental  requirements  for  correlation  using  any  of 

the  methods  of  the  next  section  is  that  the  two  TV  images  be  preprocessed 

such  that  the  scenes  have  the  same  spatial  resolution.  Since  both  TV 

systems  have  480  active  TV  line;,  the  PTS  will  have  4 times  as  many 

lines  on  a given  target  as  the  niissile  seeker  TV  because  of  the  1:4  FOV 

ratio  as  shown  in  Figures  2 and  3.  Since  the  resolution  of  the  seeker 
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TV  cannot  b<!  Increased,  the  obvious  solution  Is  to  decrease  the  PIS 
resolution  by  a factor  of  four.  This  can  be  acccoipllshed  by  averaging 
the  PTS  video  as  shoim  In  Chapter  III.  For  the  remainder  of  this  chapter 
It  trill  be  assumed  that  the  PTS  Image,  henceforth  referred  to  as  the 
high  resolution  (HR)  Image,  and  the  seeker  Image,  henceforth  referred 
to  as  the  lov  resolution  (LR)  Image  have  been  preprocessed  such  that 
they  have  tbt  same  spatial  resolution  prior  to  application  of  any 
correlation  algorithm. 


Figure  2.  Target  space  projection  of  narrow  & wMe  FOV  images. 
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After  the  resolutions  of  the  two  Images  are  equalized  a number  of 
correlation  or  matching  methods  can  be  investigated.  For  the  remainder 
of  this  report  the  dimensional  relationships  between  the  two  images 
will  be  as  shown  in  Ficjure  4. 


1 

i 

Figure  4.  K x L HR  image  located  at  position  (p,q)  of  N x M LR  image.  j 

i 

The  missile  seeker  image  is  represented  by  a N X M array  of  points. 

The  values  of  N and  M would  be  determined  from  a choice  of  sampling 
rate  and  number  of  TV  lines  of  the  missile  seeker  system.  The  PTS  image 
or  HR  image  is  represented  by  a K X L array  of  points  in  Figure  4.  This 
K X L array  might  be  all  or  only  a portion,  containing  the  target,  of 
the  PTS  image  after  its  resolution  has  been  changed  to  that  of  the  missile 
seeker  image.  The  p and  q dimensions  in  Figure  4 give  the  vertical  and 
horizontal  position  of  the  HR  image  in  the  LR  image.  These  indices 
start  in  the  upper  left  corner  of  the  LR  image,  where  p = q = 0. 
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B.  Image  Comparison  Methods 

For  digital  matching  of  two  Images  a number  of  alternatives 
exist.  In  this  section  four  candidate  algorithms  are  presented  and 
explained.  A detailed  comparison  of  these  methods  Is  given  in 
Section  C of  this  chapter. 

1.  Direct  Method 

The  classical  approach  to  the  problem  of  determining  where  twc 
signals  match  Is  correlation.  Consider  two  functions  f^  (t)  and 
fg  (t)  as  shown  In  Figure  5.  The  correlation  integral  is  defined  to 

be 

C(T)  -f  f,(t)  f,(t  . T)<lt  (1) 

where  T Is  allowed  to  take  on  values  between  -»  and  +<».  '^he  value  of 
T which  maximizes  C(t)  in  Equation  1 is  the  correlation  peak  and  is 
defined  to  be  the  match  point  between  the  two  signals.  In  Figure  5 
this  will  occur  when  t = t£  - t^.  It  Is  obvious  that  determining  the 
correlation  peak  consists  of  multiplying  one  signal  by  the  other  signal 
shifted  by  t and  then  evaluating  the  area  under  the  resulting  curve. 


Figure  5.  Two  functioris  to  be  correlated. 

If  both  signals  are  sampled  and  held,  then  Equation  1 can  be 

approximated  by  the  following  equation. 

00 

C{p)  = T I f^(n)f2(n  + p)  (2) 

n=“oo 

where  f^(n)  = f^(nT)  and  f2(n  + p)  = f2(nT  + pT)  and  T is  the  sampling 
interval.  As  T becomes  very  small,  the  result  in  Equation  2 approaches 
that  of  Equation  1 where  the  p which  maximizes  C(p)  is  defined  as  the 
correlation  point. 


~ • _/~ : 
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The  tMO  TV  Images  arr  first  sampled  and  preprocessed  to  match 
spatial  resolution  and  then  stored  In  arrays.  The  HR  Image  Is  a K X L 
array  and  the  LR  Image  is  a N X M array  as  shown  in  Figure  4.  Therefore 
a two-dimensional  discrete  correlation  algorithm  is  given  by 

R(p.q)  • ii  ^ HR(n,m)  LR(n+p,m+q) 

^ n«l  m^l 

for  0 £ p N - < 

0 < q < M - L (3) 

where  R(p,q)  Is  the  correlation  function,  and  the  division  by  KL 
Is  a scaling  factor.  Equation  3 Is  referred  to  as  the  Direct  Method  in 
this  report. 

Using  the  algorithm  of  Equation  3,  the  selected  K X L array  of 
HR  points  Is  compared  to  each  array  of  LR  points  of  dimension  XXL 
In  the  total  N X M LR  array.  The  algorithm  produces  the  correlation 
array  R(p,q).  In  most  situations  the  maximum  value  of  the  correlation 
function  Indicates  image  registration  or  match.  However,  in  tha 
present  case  since  the  LR  image  spans  a wider  field-of-view,  R(p,q) 
is  actually  a cross-cjrrelation,  and  therefore  it  is  possible  that  the 
maximum  value  of  the  correlation  functior  does  not  indicate  a target 
match  between  the  HR  and  LR  image.  In  order  that  the  maximum  value 
of  the  correlaiion  function  indicate  target  location  in  the  LR  image, 
both  image  array  values  need  to  be  normalized.  Once  normalization  is 
accomplished,  the  calculation  of  correlation  points  is  insensitive  to 
magnitude  values  and  depends  on  the  pattern  of  data  points  found  in 
the  K X L arrays  of  the  LR  image.  Therefore,  in  order  to  use  Iho  Direct 
Method,  image  normalization  must  be  implemented.  Image  normalization  is 
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discussed  in  further  detail  in  Section  C of  this  chapter. 

2.  Fast  Fourier  Transform  (FFT)  Method 

Because  of  the  transfons!  relationship  between  correlation  and 
power  spectral  density,  the  cross  correlation  between  the  HR  and  LR 
images  is  given  by 

R(p,q)  = IFR  {|FFT  rHR(n.m)]  FFT  [LR(n.m)31} 

= IFFT  {G(k,l)}  (4) 

where  FR  [ • ] is  the  fast  fourier  transform,  IFFT  is  the  inverse  fast 
fourier  transform,  G(k,l)  is  the  cross  power  spectral  density  of  HR 
and  LR  and  R(p,q)  is  the  cross  correlation  function.  In  Equation  4 
both  the  HR  and  LR  arrays  have  to  be  the  same  size  and  preferably 
be  a power  of  two.  To  accomplish  this  both  arrays  are  filled  out  with 
zeroes  to  form  512  x 512  arrays.  The  result  yields  512  x 512  values  of 
'1(p,q). 

. Phase  Correlation  Method 

A major  problem  in  correlating  two  different  size  images  is  that  the 
resultant  correlation  function  is  in  fact  a cross-correlation  instead  of 
the  desired  auto-correlation.  The  auto-correlation  function  has  its 
maximum  value  for  zero  relative  shift;  this  is  not  necessarily  true  for 
the  cross-correlation  function.  Thus  it  is  quite  possible  that  the 
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cross-correlation  function  will  have  false  peaks  which  are  larger  than 
the  p(*alc  for  zero  shift.  The  false  peak  condition  can  occur  quite  easily 
for  portions  of  the  LR  Image  of  high  brightness,  especially  If  the 
smaller  HR  Image  Is  of  lesser  brightness.  Therefore,  the  problem  is 
one  which  depends  on  relative  magnitudes  of  the  various  sub-elements  of 
the  larger  image. 

One  technique  that  reduces  magnitude  effects  in  '"‘‘c  ;,ross -correlation 
function  Is  phase  correlation.  The  phase  correlation  i\.nv.t'>on  between  the 
HR  and  LR  Images  is  given  by 

R(p,q)  = IFFT  t>^fT[HR(n,m)}  tFFT[LR(n,m])* 

|fFFT7HR(n,m)]}  {FFT[LR(n.m)]*j 

where  * denotes  the  conjugate. 

The  phase  correlation  method  Is  essentially  a poInt-by-poInt 
normalization  In  the  frequency  domain  of  the  cross-power  spectrum. 

The  same  requirements  on  the  size  of  the  HR  and  LR  arrays  exists  as 
for  the  FFT  method. 


Another  algorithm  for  determining  the  similarity  and  amount  of 
registration  between  two  images  is  known  as  a Sequential  Similarity 
Detection  Algorithm  (SSDA).  Consider  a K X L HR  image  (reference 
image)  and  a N X M LR  image.  The  SSDA  registration  surface  is 
cotriputcd  by  adding  the  absolute  value  of  the  difference  between  the 


HR  and  LK  pixels  for  each  value  of  p and  q as  given  by  Equation  6. 


(6) 


E(p.q)  |HR(n,m)  - LR  (n+p,  mfq)| 

for  0 < p < N - K 
0 < q £ M - L 

The  registration  point  (denoted  by  p*,  q*)  is  the  value  of  p and  q for 
which  E is  a minimum.  The  advantage  of  this  method  over  the  Direct 
Method  is  that  subtraction  circuits  rather  than  multiplication  circuits 
are  needed  for  implementation.  It  should  be  pointed  out,  however,  that 
the  SSDA  method  is  not  a correlation  technique  since  Equation  6 is 
not  related  to  Equation  1,  the  correlation  integral.  A further  corsparison 
of  these  two  methods  is  given  in  later  sections  of  this  report. 

C.  Analysis  of  Image  Comparison  Methods 
1.  Image  Normalization 

As  stated  in  Section  B of  this  chapter  it  is  necessary  to  normalize 
the  correlation  algorithms  so  that  false  peaks  are  less  likely  to  occur. 
False  peaks  can  occur  for  the  following  reasons: 

a.  For  correlation  computations  using  the  Fast  Fourier  Transform 
(FFT),  the  two  arrays  must  be  the  same  size.  To  accomplish  this  the 
HR  array  is  filled  in  with  zeroes  to  make  it  the  same  size  as  the  LR 
array  (array  sizes  must  be  a power  of  two).  Therefore  the  two  images 
being  correlated  are  not  identical  which  means  the  computation  is  a 
cross-correlation  rather  than  an  auto-correlation.  There  is  more  of  a 
possibility  for  false  peaks  (any  peak  which  does  not  correspond  to 
correct  registration)  in  cross-correlation  than  in  auto-correlation. 
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b.  The  size  of  the  target  In  the  HR  array  may  be  slightly 
different  from  Its  size  In  the  LR  array.  This  could  be  caused 

by  the  ratio  of  the  field  of  views  (FOV)  being  slightly  different 
from  the  assumed  ratio  or  by  the  FOV  ratio  not  being  an  Integer 
number.  This  reduces  the  magnitude  of  the  peak  at  the  actual 
correlation  point  which  could  result  In  an  erroneous  decision. 

c.  Contrast  differences  In  the  two  TV  signals  could  cause  a 
false  peak. 

d.  Differences  In  Illumination  levels  can  cause  false  peaks. 

To  Illustrate  this  consider  the  Direct  Method  given  In  Equation  3. 

In  the  Ideal  case  of  exact  correlation  each  LR  pixel  Is  exactly 
equal  to  the  corresponding  HR  pixel 

k ' 

R(p»,  ,.)  .1  II  hr2;  t.iti)  (7) 

w.  n*l  m=l 

where  p*  and  q*  denote  the  coordinates  for  perfect  registration. 

For  this  Ideal  case  consider  a non-matching  coordinate  point  p,  q 
where 

LR(n+p,m+q)  = HR(n,m)  = (8) 

Then 

It  is  easily  seen  that  even  in  the  perfect  registration  case,  a search 
for  the  peak  value  of  R(p,q)  does  not  necessarily  yield  the  correct 
registration  point.  For  the  Direct  Method,  normalization  can  be 
accomplished  as  follows: 
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R(p.q) 


k L 2 

Z Z HR(n.m)LR(rH-p,mfq) 

.n»l  ni«l  . 


• k L 1 r 9 

l z HR  (n,m)  i i LR  (n+p,nH^q) 
.n»l  m*l  J Ln»l  m*l 


This  obviously  Involves  considerable  more  computation  time  than  the 
unnormal Ized  Direct  Method  cf  Equation  3. 

For  the  FFT  Method,  normalization  can  be  accomplished  by  evaluating 
the  IFFT  of  the  phase  correlation  function  as  outlined  In  Section  B-3  of 
this  chapter. 

The  Sequential  Similarity  Detection  Algorithm  (SSDA)  can  be  normalized 

I 

by  changing  Equation  6 In  Section  B-4  of  this  chapter  to 


k L , 

E(P»q)  - i T lHR(n,m)  - M - LR(n+p,  m+q)  + LR(p,q)|  (H) 
n*l  m=l 


for  0 1 P 1 

0 < q < M-L 

where  LR(p,q)  and  HR  are  average  shades  of  grey  and  are  given  by 
- 1 

LR(p.q)  = rr  ^ ^ LR(n+p,  nH-q) 

n=l  m=l 


- 1 L 

HR  = ir  z r HR(n,m) 

n=1 
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2.  Problews  Associated  with  FFT  and  Phase  Correlation  Methods 

Of  the  four  Methods  presented  In  Section  B,  the  FFT  and  Phase 
Correlation  Methods  are  the  Most  difficult  to  ImpleMent.  A number 
of  factors  Involved  with  these  two  methods  make  them  somewhat  less 
than  prime  candidates  for  the  hand-off  problem.  Both  methods  require 
that  the  Input  arrays  be  square  and  that  the  dimension  should  be  a 
power  of  two.  As  shown  In  Section  A»  the  HR  Image  Is  preprocessed 
because  of  Its  smaller  FOV  such  that  It  Is  no  longer  the  same 
dimension  as  the  LR  array.  Therefore  It  would  be  necessary  to  fill  the 
HR  array  with  zeroes  In  order  to  make  It  the  same  size  as  the  LR 
array. 

Since  It  Is  not  known  where  the  target  may  lie  In  the  LR  Image. 

It  Is  required  that  the  entire  LR  array  be  stored  when  using  the 
FFT  or  Phase  Correlation  methods.  The  HR  Image  also  has  to  be  stored 
In  memory.  Additional  memory  Is  also  needed  for  the  Intermediate 
steps  of  these  methods.  The  Dlri^t  Method  and  SSDA  methods  can  be 
used  with  only  a small  percentage  of  the  memory  needed  for  the  FFT 
and  Phase  Correlation  methods.  Also,  both  the  FFT  and  Phase  Correlation 
algorithms  involve,  at  one  stepv  complex  multiplications  of  two- 
dimensional  arrays.  This  again  would  add  greatly  to  the  hardware  needed 
for  their  Impleinentatlon  that  Is  not  needed  for  the  direct  or  SSDA  methods. 
Because  of  the  Increased  amount  of  hardware  required  and  the  fact  that 
the  FFT  and  Phase  Correlation  methods  cannot  be  impleinented  in  real  time, 
the  FFf  and  Phase  Correlation  methods  were  eliminated  as  candidate  algorithms. 
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3.  Quantization  of  the  Input  Video  Signals 

All  the  algorithms  discussed  In  Section  B of  this  chapter  require 
that  both  the  HR  and  LR  video  signals  be  digitized.  The  process  of 
digitizing  continuous  signals  can  be  thought  of  as  two  separate  steps. 
The  first  Is  sampling  at  discrete  Instants  cf  time  and  the  second  Is 
quantization.  A discussion  of  the  selection  of  an  appropriate  sampling 
rate  will  be  given  In  Chapters  3 and  4 of  this 'report.  In  this  section 
the  different  quantizers  Investigated  are  discussed  and  the  trade-offs 
used  In  choosing  a particular  quantizer  are  given. 

The  input-output  relationship  for  the  four  basic  quantizers  con- 
sidered in  this  section  are  shown  in  Figure  6.  Figure  6 (a)  is  the 
multi  bit  quantizer  or  one  that  has  a very  large  number  of  quantization 
levels.  Since  there  are  many  fairly  Inexpensive  commercially  available 
8 bit  A/D  converters  capable  of  operating  at.  10  MHz,  they  will  be  con- 
sidered as  multibit  quantizers. 

Figure  6 (b)  gives  the  input-output  relationship  of  a tvo  level 
or  one-bit  quantizer.  The  weight  of  the  output  is  +1  if  the  input  is 
above  the  mean  voltage  level  and  is  -1  if  the  input  is  below  the  mean 
voltage  level.  With  this  quantizer  a means  must  be  provided  for 
finding  the  average  shade  of  grey  of  the  K X L high  resolution  reference 
image  and  of  each  of  the  K X L sub-arrays  of  the  low  resolution  image 
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corresponding  to  eich  set  of  p and  q values  In  Equation  3.  This 
could  be  done  by  quantizing  the  Images  irtth  an  8 bit  A/D  ccnverter, 
co^Nitlng  the  average  values  In  the  host  coi^Hiter,  during  one  frame, 
and  using  these  computed  average  values  for  one-bit  quantization 
during  the  following  frame.  The  average  signal  level  Is  not  expected 
to  change  significantly  from  frame-to-frame.  If  the  average  signal 
level  does  change  significantly  from  frame-to-frame,  some  form  of 
adaptive  average  computation  should  be  employed.  A sub-optimal  but 
real-time  Implementable  solution  to  this  problem  Is  given  In  Chapter 
3. 

Figure  6 (c)  Is  the  Input-output  relationship  for  a four  level  or 
two-bit  q^jantlzer.  Assuming  the  mean  and  standard  deviation  of  the 
Input  signal  are  known,  there  are  two  parameters,  "a"  and  "v^/a",  which 
can  be  chosen  so  as  to  minimize  the  variance  of  the  estimate  of 
R (l.i.,  minimize  the  loss  In  S/N  ratio  due  to  quantization). 

Figure  6 (d)  Is  the  Input-output  relationship  for  a three  level 
quantizer.  The  advantage  of  this  quantizer  over  the  two-bit  quantizer 
Is  that  the  weighting  of  the  products  In  Equation  3 are  either  + 1 or 
0 which  leads  to  a simpler  circuit  realization.  The  disadvantage  Is  that 
the  loss  In  S/N  ratio  Is  greater  than  for  the  two-bit  quantizer.  The 
mean  and  standard  deviation  (rms  value)  of  the  input  video  must  be 
calculated  for  this  method. 

The  selection  of  the  optimum  parameters,  a and  v^/o,  is  made  from 
an  analysis  of  the  degradation, In  the  mean  square  S/N  ratio.  This 
analysis  is  presented  In  Section  C-5  of  this  chapter. 
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a)  MuUibIt  quantizer  b)  One-bit  quantizer 


c)  Two-bit  quantizer  d)  Three  level  quantizer 


Figure  6.  Quantizers 

Higher  order  quantizers  are  not  considered  since  they  would  require 
larger  adders  with  larger  computation  times  thereby  leading  to  smaller 
arrays  for  real-time  correlation.  It  will  be  shown  in  a later  section 
that  the  S/N  ratio  gain  from  using  a larger  reference  array  is  greater 
than  the  gain  using  higher-order  logic  in  the  quantizers  in  a smaller 
reference  array. 
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4.  Iiaqe  Honwallzation  Using  the  One-Bit  Quantizer 

It  Is  shOMn  In  Section  C>1  that  the  Direct  Method  requires 
norMlIzatlon  to  prevent  false  peaks.  A method  of  normalization 
Is  given  In  Equation  10.  When  using  the  transfer  function  shown  In 
Figure  6 (b)  to  quantize  both  the  HR  and  LR  signals,  the  outputs 
are  always  + 1.  Under  this  situation  the  denominator  term  of  Equation 
10  reduces  to  (KL)^.  Then  Equation  10  becomes 

R(p»q)  ° - A r E E HR{n,m)LR(n+p,m+q)l  ^ (13) 

(KL)2  Ln=1  m=l  J 

Equation  13  Is  the  normalized  direct  cross-correlation  expression  for 
one-bit  by  one-bit  quantization.  The  square  root  of  Equation  13  would 
also  be  a normalized  estimate  of  the  cross  correlation  and  Is  seen  to  be 
equivalent  to  Equation  3.  Therefore  the  one-bit  by  one-bit  two- 
dimensional  correlation  algorithm  given  by  Equation  3 is  normalized. 

Its  peak  value,  which  occurs  at  complete  registration,  is  one.  No 
additibnal  hardware  dedicated  to  the  normalization  process  is  required. 

5.  Quantization  Effects  on  S/N  Ratio  for  the  Direct  and  SSDA 
Correlation  Methods 

A detailed  analytical  study  was  performed  so  that  the  effect  on  the 
mean  square  S/N  ratio  due  to  different  quantizers  being  used  on  the  HR 
and  LR  signals  could  be  determined.  The  analysis  was  done  for  the 
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quantizers  shown  in  Figure  6 and  also  for  the  six  level  quantizer  shown 
In  Figure  7. 


Mgure  7.  Six  level  quantizer. 


In  this  section  the  variance  derivations  for  the  Direct  and  SSDA  methods 
using  one-bit  quantization  are  presented.  The  results  for  the  other  quanti- 
zers are  summarized  snd  a comparison  of  the  two  methods  is  presented. 


a.  Direct  Method 

Consider  the  estimation  of  the  Direct  Cross-Correlation  function 
for  the  first  line  of  the  high  resolution  image  with  the  first  line  of 
the  low  resolution  image  as  shown  in  Figure  8.  The  estimation  of  the 


Figure  8.  Layout  for  correlation  of  first  line  of  two  images. 
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Direct  Cross-Correlation  function  for  the  first  line  Is  given  by 

R(l.q)-f  I HRO.m)LRO.  «+q)  (14) 

n*! 

where  HR  and  LR  are  the  high  resolution  and  low  resolution  Images 
respectively.  To  simplify  the  notation,  the  first  row  designation  can 
be  dropped  yielding 

R(q)  » f I :JR(ir.)I.R(m^)  (15) 

^ m»l 

If  the  sampling  rate  Is  less  than  or  equal  to  the  Nyquist  rate,  the 
adjacent  samples  can  be  considered  to  be  Independent.  Therefore, 
except  for  the  three  or  four  values  of  q close  to  registration,  the 
correlation  coefficient  between  the  two  Images  Is  small  (i.e. , p « 1). 
If  the  two  Images  are  quantized  with  quantizers  g^  and  g^  respectively, 
then  Equation  15  becomes 

R(q)  “ t I g^[HR(m)]g2[LR(m+q)]  (16) 

m*l 

The  variance  of  this  estimate  Is  given  by 

07) 

R 

A 

where  <R>  denotes  the  expected  value  of  R given  by 
<R>  * ^ |u  gi[HR(m)3g2[LR(m+q)]| 


(18) 
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KoMever,  because  of  the  assumed  Independence  of  the  samples,  Equation 


18  reduces  to 


<R>  = ^ I^<^g^[HR(m)]  g2[LR(n»+q)3^  (19) 


The  first  term  in  Equation  17  can  be  written  as 
A L L 

<R  > I I gi[HR(m)3g2[LR(m+q)]g^[HR(n)] 

\L  n=l  m=l 

(20) 


Separating  the  term  for  which  n=m  in  Equation  20  yields 


<R^>  * \ I <^/[HR(m)3g2^f'.R(in+q)3 


L*-  m=l 


1 


L L 


\ 


+ i I /g^[HR(m)3g2[LR(m+q)3g^[HR(n)3g2[LR(n+q)3y 


n=l  m=l  \ • 
mj«n 


(21) 


Consider  the  case  where  both  quantizers  g^  and  g2  have  an  input/ 
output  relationship  as  shown  in  Figure  6{b),  then  Equation  19  becomes 


CO  00 


L r ^ ^ 

<//  {l)f(HR.LR)d(HR)d(LR)  +J  J 

-<»-<«>  0 0 


0) 


0 so 

f(HR,LR)d(HR)d(LR)  ^ f f (-1 )f (HR,LR)d{HR)d(LR)  + 

-so  0 


(-l)f(HR,LR)d(HR)d(LR) 


(22) 
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where  f(KR,LR)  Is  the  jcirit  density  function  of  the  HR  and  LR  Images. 

If  f(HR,LR)  Is  symetrlc  with  zero  mean,  then  Egfuatlon  22  reduces  to 

<n>  « f I C(l)(l/4)  + (1)0/4)  + (-1)0/4)  + (-1)0/4)] 

'■  m*l 

or  <R>  « 0 (23) 

Because  of  the  assumed  Independence  of  the  samples  and  the  Input/ 
output  relationship  of  Figure  6(b),  the  first  term  In  Equation  -21  reduces 
to 

T I,  nxD-f  (24) 

r m«l 

and  the  second  term  In  Equation  21  becomes 

(L)(L-1 ) <g,[HR]y  <^2lLR])>  ^ . 0 (25) 


Substituting  Equations  23,  24,  and  25  Into  Equation  17  yields 


Equation  26  Is  an  expression  for  the  variance  of  the  estimate 

of  R when  the  Input  1s  quantized  into  two  levels.  To  obtain  the 

variance  of  the  estimate  of  the  true  correlation  of  the  two  signals, 

2 

, the  relationship  between  R and  p must  be  established.  If  the 
Input/output  relationship  for  the  quantized  high  resolution  and 
quantized  low  resolution  video  signals  are  given  by  g^(HR)  and  g2(LR) 


respectively,  where  and  92  are  both  as  shown  In  Figure  6(b),  then 
R(q) ‘^iCHRlggCLRCq)]^  (27) 

where  the  HR  and  LR(q)  signals  are  jointly  norml  with  covariance  y 
given  by 

u=  <HRLR(q))>  - <HR><LR(q))>  (28) 

and  correlation  coefficient  p given  by 


p 


-Ji 

*^HR‘^LR 


(29) 


If  X and  Y are  jointly  normal  random  variables  with  covariance  y 
and  the  random  variable  Z Is  formed  by  Z » g(X»  Y)  where  g(x,y)  Is  an 
arbitrary  function,  then 


E{Z}  » E{g(X,Y)} 


g(x,y)f(x,y)dxdy 


(30) 


where  f(x,y)  is  the  bivariate  distribution  function  for  the  random 
variables  X and  Y.  Price's  theorem  [2],  [3]  states  that 
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Applying  Price's  theorem  with  n»l  to  the  expression  for  R(q)  In  Equation 
27  yields 


3R(q)  I 

3y(qt " 3vlq)  ^ \ aHR  3LR(q)  ) ™ 


and  from  Equation  29. 

3R(q)  , 3R(q)  . 3y(q)  , 3R(q| 
3^7  3p(q)  3p(q)  3y(q) 


°HR  °LR 


(33) 


or 


3^  “ ‘^HR  ®LR  ^ 1 1 

^3g^[HR]^  / 3g2[LR(q)]^| 
^ 3HR  } )\ 

(34) 

However  from  Figure  6(b) 

^ = 26[HR]  and 

ag, 

jS  - 

(35) 

and  Equation  34  reduces  to 


00  ^00 


3E!al  = _J — f f 

3p(9)  J — T J.CO  J- 


26[HR]26[LR]  Exp 


[ 


2(1 


U( 


HR^  _ 2pHRLR 
'"HR  '^LR 


.tBL\l 


d(HR)d(LR) 


or 


(36) 
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wiiere  It  has  been  assumed  that  HR  and  LR  are  jointly  gausslan.  For  the 
case  being  considered  p « 1 which  leads  to 


From  Equation  37 


(38) 


(39) 


using  Equation  26. 

If  one  uses  the  two  level  or  one-bit  quantizer  with  input/output 
relationship  as  shown  in  Figure  9 and  repents  the  steps  given  above,  one 
finds  the  variance  of  this  correlator  to  be 

o/  = 3 (4)(1)  (40) 


or  a variance  which  is  exactly  three  times  that  obtained  using  the  one- 
bit  quantizer  described  by  Figure  6(b),  If  the  quantizer  in  Figure 
6(b)  quantizes  to  levels  of  ±n,  repeating  the  steps  above  reveals  that 
the  variance  of  the  resulting  correlator  is  again  given  by  Equation  39. 

i Output 


1 


Input 


Figure  9,  AUernate  one-bit  quantizer. 
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The  products  for  the  correlator  described  by  Figure  6(b)  are  either 
-1  or  +1  as  shown  In  Figure  10(a).  The  correlator  described  by  Figure  9 
with  an  Input/output  table  given  In  Figure  10(b)  yields  an  estimate  of  r 
with  a variance  three  times  as  large  as  the  correlator  In  Figure  10(a). 

If  one  biases  the  output  products  of  the  correlator  described  in  Figure 
10(a)  as  shown  In  Figure  10(c),  the  mean  of  the  estviiate  changes  but  the 
variance  does  not.  The  correlator  In  Figure  10(c)  is  therefore  equivalent 
to  that  of  Figure  10(a).  Now  if  one  Implements  the  correlator  In  Figure 
10(d),  the  resulting  variance  is  the  same  as  the  correlator  in  Figure 
10(c)  but  the  counter  can  be  designed  to  count  only  half  as  high  thereby 
simplifying  the  implementation.  The  correlator  with  input/output  rela- 
tionship  described  in  Figure  10(d)  Is  easier  to  implement  than  the  one 


described  in  Figure  10(a)  since  a ripple  counter  rather  than  an  up-down 


counter  can  be  used.  In  addition  the  input/output  relationship  described 


in  Figure  10(d)  can  be  realized  with  exclusive  OR  gates. 


39 


b.  SSDA  Method 

Consider  the  estimation  of  the  I'SM  function  for  the  first  line  of 
the  high  re»«Vjt1on  Image  with  the  first  line  of  the  low  resolution 
Image  as  shown  In  Figure  8.  The  estimation  equation  for  the  SSOA 
function  Is  given  by  Equation  41  where  Iffi  and  LR  refer  to  high  resolution 
and  low  resolution  Images  respectively. 

RO.q)-f  I !HR(l.m)  - LRO.  m+q)|  (41) 

To  simplify  notation,  the  first  row  designation  can  be  dropped,  then 
Equation  41  becomes 

. 1 L 

R(q)“f  I |HR(m)  - LR(m+q)|  (42) 

m*l 

Assume  that  the  sampling  rate  is  less  than  or  equal  to  the  Nyquist 
rate.  This  means  that  adjacent  samples  are  almost  independent  for  all 
values  of  q except  for  values  near  the  point  of  registration.  Therefore 
for  all  cases,  except  the  case  near  registration,  the  correlation 
coefficient,  p,  is  small  and  assumed  to  be  much  less  than  one. 

If  the  two  sampled  lines  of  video  are  quantized  with  quantizers  g^ 
and  g2  respectively,  then  Equation  42  is  written  as 

R(q)  = L ^ |giCHR(m)]  - g2[LR(m+q)]| 


(43) 
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"-TWWfieis  0rdmeim~qumil4^^  can  be  Investigated  by 

considering  the  cases  of  q for  nhlch  the  saaples  can  be  asstned  Inde- 
pendent. The  variance  of  the  estlnato  of  R Is  given  b> 

- <R^>  - <R>^  (44) 

R 

Assune  that  the  quantizing  functions  and  g2  are  given  as  In  Figure  9. 

A 

Consider  first  <R>,  given  by  Equation  45. 


<R>  *^|u  l9]tHR(m)]  - g2[LR(m+q)]|j>  (45) 

Since  the  samples  are  Independent,  Equation  45  becomes 

<R>  - f I <!gi[HR(m)]  - g2[LR(m+q)]|>  (46) 

m*l 


FIgurell  shows  the  possible  outcomes  of  the  function  |g^[HR(m)]  - g2[LR(nH^)]| 
for  all  combinations  that  the  values  and  g2  can  possess. 


g2[LR(m+q)] 


0 

1 


q^[HR(m)] 
0 1 


0 

1 

1 

0 

Figure  11.  Outcomes  of  (g^  - g2l. 


Using  the  results  of  FIgurell  and  expanding  the  expected  value. 
Equation  46  becomes 


<R>  » r 


n / / (l)f(HR.LRAl(HR)d(LR) 

‘■wil-'—  Jo 


u: 


(*»/j 


(l)f{HR.LR)d(HR)d(LR) 


Assuriing  HR  and  LR  to  be  jointly  gausslan  and  assuming  Independent 
samples*  Equation  47  yields. 


<«>  • I j,  >/.X  f(HR.LR)d(HR)d(LR)| 


<R>  « f (L)(2)  (I)  = i 


Therefore,  the  second  term  in  Equation  44  is 


D 2 1 

<R>  = ^ 


Next  consider  <R  > given  by  Equation  50 


=<; 


L L 


I I |gi[HR(m)]  - 92[LR(m+q)]|' 

L m=l  n=l 


g^[HR(n)3  - g2[LR(n+q)]| 


> 


(48) 


(49] 


(50] 


Separating  terms  for  which  m=n  in  Equation  50  yields 
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<i?»  ■ ^ J <{|9, [«(«)]  - g2CU(«>»<i)]|*}> 


L L 


|9l[HR(n)]-  92CLR(n+q)]lJ^ 


Again  using  the  results  of  Figure  11  and  the  assumptions  made  previously, 


Equation  51  becomes 


-■ft], If/: 


(1)f(HR,LR)d(HR)d(LR)  + 


• 0 

f J (l)f(HR,LR)d(HR)d(LR)  t 1-  J I 

mi^n 

/■  / (1)f(HR,LR)d(HR)d(LR)  + / / (1)f(HR,LR)d(HR)d(LR) 

-00  0 0 -00 


\f  f 

\o: 


(1)f(HR,LR)d(HR)d(LR)  + 


0&  I 

/.  / 


(1)f{HR,LR)d(HR)d(LR) 


Equation  52  reduces  to. 


<r2>.  (D(2)(l)  *^^)(l)(L-1)(2)(|)(2)(1) 


1 a (L-,)  1 =La 


(53) 
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Substituting  Equations  49  and  S3  Into  44  yisids 

2 . 1 4.  1 


(54) 


Of  Importance  here  Is  to  f1r<d  out  how  the  mean  square  Signal -to-Nolse 

(ms  S/N)  ratio  Is  degraded  by  the  use  of  different  quantizers.  .The 

2 

variance  of  the  estimate  of  p,  o , directly  reflects  the  degradation. 

p 

The  variance  of  p Is  found  as  follows. 

Let 

r{q)  - E{|g^[HR(m)]  - g2[LR(fiHq )J1 } (55) 


Assume  that  HR  and  LR  have  the  joint  gausslan  density  function  given  by 
Equation  56 


f(HR,LR) 


2pHRLR  ^ 
•^HR®LR 


(56) 


Then  Equation  55  is  written  as 


r{q)  = 


1 


d(HR)d(LR) 


y*  f |gi[HR(m)]  - g2[LR(m+q)]| 
-00  *^—00 

(57) 


Us 


ing  the  results  of  Figure  11  and  assuming  p 


2 


<<  1,  Equation  57  becomes 


r(q) 


^*®HR®LR 


fl  / HR^  . 


a. 


^ LR^  _ 2pHRlR 
®HR®LR 


i 


•a 


•C  ] 


d(HR)d(LR) 


d(HR)d(LR) 

(58) 


Using  the  expansion  on  for  small  x*  Equation  58  becomes 


Evaluating  Equation  59  yields 
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r(q)  • j*|-(-l+0)(W) 

r(q)  • I - J 
or 

p ■ j - t r(q)  (60) 

Therefore  from  the  relationship  of  Equation  60  It  Is  shown  that 

222  /Cl \ 

0.  • w cr  (61) 

P ft 

Substituting  Equation  54  Into  61  yields 

2.2'  . . 2.467 

% ”•  2T  " (?)  r *—"■ 

If  one  uses  the  two  level  or  one-bit  quantizer  with  Input/outout 
relationship  as  shown  In  Figure  6(b)  the  variance  in  the  estimate  of  p 
Is  the  same  as  that  given  In  Equation  62.  Therefore,  If  the  SSDA  method 
is  to  be  Implemented  with  a one-bit  quantizer,  the  choice  of  quantizer 
depends  on  which  is  easier  to  implement. 
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c.  Results  of  S/N  Ritlo  Analysis  for  Higher  Order  Quantizers 

The  analysis  of  the  degradation  of  the  S/N  ratio  mss  extended  for 
both  the  Direct  and  SSOA  Methods  to  the  cases  Mhere  three,  four,  and  six 

level  quantizers  are  used.  As  shoMn  In  Figures  6(c) » 6(d).  and  7.  the 

2 

variables  v^/o.  v^/o.  V2/0.  a and  b are  chosen  such  that  Is  mlnliiilzed. 
Table  1 shows  the  results  of  this  analysis  for  both  the  SSOA  and  Direct 
methods.  The  curves  presented  In  Figure  12  show 


Direct  Method  | 

1 SSDA  Method  ] 

No 

9 

Optimum 

2 

Optimum 

Quantization 

Levels 

aJ*L 

P 

Parameters 

a 

P 

Parameters 

2 

2.467 

t 

1 

1 

1 

2.467 

3 

1.525 

Vq/o  * 0.6 

1.88 

WIWUBh 

4 

^0^^  * .^5 

a * 4 

HI 

■ra™™ 

6 

1 

v-j/u  * 0.6 

V2/0  * 1.4 
a = 3 
b * 6 

1.613 

V-j/o  = .3 
= .7 

a = 3 
b = 5 

Table  1.  Results  of  S/N  ratio  analysis. 

2 

Op  *L  vs.  no.  of  quantization  levels  for  the  two  image  matching  methods 
being  considered.  Figure  12  clearly  shows  that  the  Direct  Method  yields 
a lower  degradation  in  the  ms  S/N  ratio  for  all  quantizers  except  the 
two  level  quantizer.  Therefore,  the  conclusion  can  be  made  that  the 
SSDA  method  would  not  be  implemented  using  the  higher  order  quantizers 


if  the  decision  were  based  only  on  degradation  of  the  ms  S/N  ratio. 
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HoMtver,  since  the  SSM  nethod  cen  be  Inplenented  with  less  and  faster 
hardwire  than  the  Direct  Method  when  wore  than  two  levels  are  used.  It 
should  not  be  ruled  out  as  a candidate  correlation  alqorlthn. 


No.  of  Quantization  Levels 

Figure  12.  Comparison  of  the  SSDA  and  Direct  Double  Summation  Methods 
From  Table  1 and  Figure  12  one  can  draw  some  conclusions  about 
correlator  design  parameters  (eg.,  reference  array  size,  number  of 
quantization  levels,  and  statistical  accuracy).  From  Table  1 the 
degradation  in  the  mean  square  S/N  ratio  for  both  methods  using  any 
number  of  quantization  levels  Is  Inversely  proportional  to  the  number 
of  pixels  used  In  the  reference  array  (I.e. , doubling  the  number  of 
pixels  In  the  reference  array  halves  the  S/N  ratio  degradation  for  a 
given  number  of  quantization  levels).  Also  from  Figure  12  the  degra- 
dation In  the  S/N  ratio  can  be  improved  by  increasing  the  number  of 
quantization  levels.  However,  relatively  little  improvement  is  gained 
by  using  more  than  four  levels  (2-bits)  with  either  the  Direct  or  SSDA 


three,  or  four  levels. 

If  the  correlator  Is  to  operate  in  real  time,  then  the  sampling 
period  of  the  Incoming  live  video  dictates  the  amount  of  time  available 
for  each  correlation  computation.  However,  Increasing  the  size  of  the 
reference  array  and  Increasing  the  number  of  quantization  levels  (eg., 
from  two  to  four  levels)  both  require  more  hardware  and  a longer  compu- 
tation time.  From  Table  1 Increasing  the  number  of  quantization  levels 
from  two  to  four  Increases  the  ms  S/N  ratio  by  2.467/1.3  or  1.898  for 
the  Direct  Method  and  by  2.467/1.715  or  1.438  for  the  SSOA  method.  The 
computations,  however,  using  four  levels  require  more  time.  In  fact  from 
data  books  on  present  state-of-the-art  hardware  one  could  more  than 
double  the  size  of  the  reference  array  with  this  Increased  computation 
time  using  two  level  quantization.  This  Increase  In  the  size  of  the 
reference  array  would  more  than  double  the  ms  S/N  ratio.  Therefore,  the 
trade-off  Is  to  make  the  reference  array  size  as  large  as  possible  for 
real-time  computation.  Because  of  hardware  considerations  It  Is  usually 
desirable  to  make  each  reference  array  dimension  a power  of  two  (i.e., 
15x16,  32x32,  32x64,  64x64,  etc.).  If  for  example  a 32x32  reference 
array  In  a two  level  correlator  can  be  computed  in  real  time  but  a 
32x64  reference  array  can  not,  then  one  might  consider  using  three  or 
four  levels  in  this  32x32  correlator  if  time  would  allow.  In  other 
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words  the  trade-off  is  to  use  as  large  a reference  arra^y  as  possible, 
then  consider  using  wore  than  two  levels  if  time  permits. 

One  additional  consideration  when  going  from  two  level  to  three  or 
four  level  quantization  should  be  pointed  out.  When  quantizing  to  two 
levels,  each  KxL  subarrity  of  the  LR  video  should  be  quantized  to  one 
level  when  above  the  nean  and  to  the  other  level  when  below  the  mean  of 
that  KxL  subarray.  For  each  value  of  p and  q In  Equation  3 or  6 all  of 
the  KxL  pixels  In  that  subarray  should  be  quantized  about  the  mean  of 
that  subarray.  Quantizing  about  some  moving  average  mean  or  about  the 
mean  of  the  entire  NxM  LR  image  leads  to  errors  in  the  correlation 
computation.  However,  since  it  is  not  practical  to  do  this,  some  trade- 
off technique  Is  usually  devised.  If  a three  or  four  level  quantizer 
Is  used, then  the  standard  deviation  as  well  as  the  mean  of  each  KxL 
subarray  should  be  comp'ited.  To  use  any  other  technique  to  find  v^/a 
in  Table  1 leads  to  errors  in  the  correlation  computation.  In  fact 
these  additional  errors  could  be  larger  than  the  gain  in  going  from  a 
two  level  to  a three  or  four  level  quantizer.  The  amount  of  error 
induced  by  inaccurate  values  of  mean  and  standard  deviation  in  the 
quantization  is  scene  dependent.  The  conclusion  is  that  if  one  uses  a 
three  or  four  level  quantizer,  care  should  be  taken  in  the  computation 
of  the  mean  and  standard  deviation  ^n  order  to  prevent  a sianificant 

X 

loss  in  the  S/N  ratio  due  to  inaccuracies  in  these  computations. 

6.  Implementation  of  the  SSDA  and  D^^ect  Methods  Using  One-Bit 
QuantizatTon 

Wiien  both  the  HR  and  LR  video  are  quantized  to  two  levels,  then 
from  Table  1 the  Direct  and  SSDA  methods  have  the  same  ms  S/N  ratio. 
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In  this  section  the  Inplanentatlon  of  the  two  methods  using  exclusive 
OR  gates  Is  shown  to  be  equivalent.  The  conclusion  Is  that  when 
quantizing  to  two  levels,,  the  SSDA  and  Direct  methods  are  equivalent. 
This  Is  not  the  case  when  one  or  both  video  signals  are  quantized  to 
more  than  two  levels.  In  this  case  the  SSDA  method  is  not  a true 
correlation  technique  while  the  Direct  method  Is. 

a.  Direct  Met^lod 

Consider  first  that  the  HR  and  LR  video  have  zero  mean  and  that 
the  HR  video  1:>  quantized  by  the  Input/output  relationship  shown  in 
Figure  9.  Then  if  the  LR  video  is  quantized  by  the  input/output 
relationship  shown  in  Figure  13,  the  correlator  inputs  when  the  input 
video  match  are  given  by  01  and  10  and  video  mismatch  are  given  by  00 
and  11  input  combinations.  If  the  output  of  the  correlator  ir.  to  be 
1 when  the  Input  video  match  and  0 when  the  input  video  mismatch,  then 
the  input/output  table  is  given  in  Figure  14  and  can  be  implemented 
using  an  exclusive  OR  (^)  gate  as  shown  in  Figure  15.  Implementation 
of  the  entire  array  is  given  in  Chapter  4.  Using  this  method,  the  peak 
value  corresponds  to  registration  of  the  two  images. 


4 

LR 

q 

1 

LR^ 

Figure  13.  Reversed  polarity  one-bit  quantizer. 
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Figure  14.  Input/output  table  for  one-bit  correlation. 


o — 


Figure  15.  Exclusive-OR  implementation. 

b.  SSDA 

The  SSDA  implementation  is  similar  to  that  given  above,  however  the 
quantizer  given  in  Figure  9 is  used  to  quantize  both  signals.  The  out- 
come of  the  SSDA  algorithm  is  shown  in  Figure  11.  The  states  00  and  11 
now  represent  matched  samples  and  01  and  10  represent  mismatch.  The 
circuit  for  implementing  the  SSDA  algorithm  is  the  same  given  in  Figure  15. 
The  adder  output  is  now  an  error  value  instead  of  a correlation  value. 
Therefore  the  minimum  value  of  the  adder  output  corresponds  to  image 
registration. 

In  this  section  it  has  been  shown  that  if  the  two  images  are 
quantized  to  two  levels  (one-bit  quantization),  then  the  Direct  Method 


Correlation 

value 
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and  SSDA  methods  are  identical.  In  fact,  either  type  correlator  design 
can  be  converted  to  the  other  by  reversing  the  polarity  of  one  of  the 
signals  before  quantization  and  looking  for  a minimum  If  originally 
looking  for  a maximum  at  the  output  of  the  correlator  (or  vice-versa). 


3.  CORRELATION  SYSTEM  CONSIDERATIONS 


The  various  trade-offs  and  other  considerations  which  lead  to  the 
correlation  implementation  in  Chapter  4 are  presented  in  this  Chapter. 

A.  Determination  of  Sampling  Frequency 
Shannon's  sampling  theorem  says  that  theoretically  all  the  infor- 
mation in  a signal  is  contained  in  the  sampled  signal  (and  can  therefore 
be  recovered  from  the  sampled  signal)  if  the  sampling  rate  is  at  least 
twice  the  bandwidth  of  the  signal  (B).  This  sampling  frequency  f^  = 2B 
is  usually  referred  to  as  the  Nyquist  rate. 

Consider  a 30  frame  per  second  525  line  TV  system  with  4:3  aspect 
ratio.  The  number  of  active  horizontal  lines  is  approximately  480.  If 
the  horizontal  and  vertical  resolution  of  each  pixel  are  to  be  the  same, 
then  (4/3)480  = 640  samples  should  be  taken  on  each  TV  line.  The  sampling 
rate  is  (4/3)(480)(525)(30)  = 10.08  MHz  or  approximately  10  MHz.  The 
sampling  period  for  a 10  MHz  sampler  is  100  nsec.  Therefore  in  order 
to  design  a real-time  correlator  each  computation  must  be  accomplished 
in  less  than  100  nsec.  In  other  words  the  value  of  R for  each  p and  q 
in  Equation  3 must  be  computed  in  less  than  100  nsec.  Howe/er,  the 
actual  bandwidth  for  most  525  line  black/white  video  signals  is  usually 
on  the  order  of  4 MHz  or  slightly  larger.  The  corresponding  Nyquist 
rate  is  approximately  8 MHz.  Since  it  is  necessary  to  sync  the  sampler 
clock  frequency  to  the  video  signal,  the  actual  sampling  rate  will  be 
slightly  less  than  8 MHz  as  explained  in  Chapter  4. 
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It  should  be  pointed  out  that  the  recommended  sampling  rate  of 
approximately  8 NHz  Is  based  entirely  on  system  bandwidth  considerations. 
Since  sampling  at  a higher  rate  will  yield  relatively  little  gain  while 
decreasing  the  available  time  for  real  time  correlation,  8 MHz  Is  seen 
as  an  upper  limit  on  the  sampling  frequency.  If  the  sampling  rate  Is  • 
decreased  In  order  to  allow  more  time  for  real  time  correlation  or  for 
shift  register  hardware  considerations  as  explained  In  Chapter  4,  then 
what  one  gives  up  Is  horizontal  pixel  resolution.  The  decreased  sampling 
rate  does  not  effect  the  vertical  pixel  resolution  since  this  Is  deter- 
mined by  the  number  of  TV  lines.  Initial  reaction  to  this  would  be  to 
conclude  that  not  much  would  be  lost  In  correlation  since  as  reported 
in  [4]  military  targets  such  as  tanks,  personnel  carriers,  jeeps,  etc., 
exhibit  a larger  spatial  frequency  content  in  the  vertical  direction 
than  in  the  horizontal  direction  (i.e. , there  is  more  high  frequency 
spatial  content  in  the  vertical  direction  than  in  the  horizontal  direttion 
for  vehicle  targets).  A much  closer  analysis  given  in  the  following 
section  reveals  that  if  the  reference  array  is  32  x 64,  most  of  the 
correlation  is  being  performed  on  the  target  background  and  not  on  the 
target  itself.  Reference  [4]  points  out  that  most  backgrounds  have 
greater  horizontal  spatial  frequency  content  than  vertical  content.  An 
obvious  example  of  this  would  be  forests.  The  effect  on  the  accuracy  of 
correlation  if  one  decreases  the  sampling  rate  from  approximately  8 MHz 
to  say  5 MHz  is  scene  dependent  and  was  not  studied  under  this  contract. 


It  is  recommended,  therefore, that  the  sampling  rate  effects  on  correlation 
be  studied  in  any  correlator  technology  program. 
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B.  Area  Correlator 

Consider  a 525  line  TV  system  with  approximately  480  active  lines 
and  a 2*  vertical  FOV.  A standard  2.3  x 2.3  meter  target  would  subtend 
6.3  lines  In  the  TV  FOV  at  5 KM,  7.9  lines  at  4 KM,  10.5  lines  at  3 KM, 
and  15.8  lines  at  2 KM.  If  the  reference  array  Is  64  x 64  (In  a 
full-frame  correlator),  then  the  target  occupies  0.967%  of  the  refer- 
ence array  area  at  5 KM,  1.52%  at  4 KM,  2.69%  at  3KM,  and  6.09% 
at  2 KM.  Therefore.  It  Is  evident  that  most  of  the  correlation  Is  being 
performed  on  the  target  background  and  not  on  the  target  Itself.  If 
the  size  of  the  reference  array  Is  32  x 32,  then  the  above  target  area 
percentages  would  Increase  by  a factor  of  four. 

The  results  of  the  above  paragraph  can  be  stated  In  a slightly 
different  way.  If  two  TV  lines  are  required  to  display  a spatial  cycle, 
then  the  highest  vertical  spatial  frequency  detectable  at  5 KM  Is  1.34 
cycles/meter,  at  4 KM  Is  1.71  cycles/meter,  at  3 KM  is  2.28  cycles/meter, 
and  at  2 KM  is  3.42  cycles/meter.  If  the  horizontal  and  vertical  reso- 
lutions are  the  same  (i.e.,  sampling  at  10  MHz)  then  the  limiting  horizontal 
spatial  ficx,::ency  resolution  is  the  same  as  the  vertical.  If  the  sampling 
rate  is  reduced  to  7.5  MHz,  then  the  limiting  horizontal  spatial 
frequencies  detectable  are  1.005,  1.28,  1.71,  and  2.57  cycles/meter 
respectively  at  the  above  ranges.  Stated  another  way,  trees  spaced 
every  0.75,  0.58,  0.44,  and  0.29  meters  apart  respectively  at  the  5,  4,  3,  2 
KM  ranges  could  be  detected  when  sampling  at  10  MHz  while  trees  spaced 
every  0,995,  0.78,  0.585,  and  0.39  meters  apart  respectively  could  be 
detected  when  sampling  at  7.5  MHz,  The  above  results  are  summarized  in 
Table  2. 
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C.  Preprocessing  of  Input  Video 
1.  High  Resolution  Image  Preprocessing 

The  purpose  of  preprocessing  the  HR  video  is  to  reduce  the  spatial 
resolution  of  the  HR  video  to  that  of  the  LR  video,  a condition  necessary 
for  correlation  purposes.  This  difference  in  resolution  is  caused  by 
the  differing  fields  of  view  (FOV),  number  of  TV  lines  per  frame,  frame 
rate,  aspect  ratio,  and  sampling  rate  of  the  two  TV  systems.  Due  to 
size,  weight,  and  cost  constraints  the  FOV  of  the  missile  seeker  will 
be  W times  (W>1)  that  of  the  FOV  of  the  fine  pointing  and  tracking 
system.  If  W is  an  integer,  then  the  preprocessor  averages  the  first 
W columns  of  the  first  W rows  of  the  HR  image  to  obtain  the  PCI,1) 
picture  element  (pixel)  of  the  reference  image.  Then  the  next  W columns 
of  the  first  W rows  are  averaged  to  obtain  the  P(l,2)  pixel  of  the 
reference  array  as  shown  in  Figure  16.  If  the  sampled  'iR  video  forms 
an  A X B array,  then  the  reference  array  is  an  (A/W)  x (B/W)  array.  The 
size  of  a possible  target  in  the  reference  array  is  then  identical  to 
the  size  of  the  target  in  the  LR  array.  Equation  63  is  the  preprocessing 
algorithm  where  W is  an  integer. 


W W 


= 4 I I [(WI  - WfM),  (WJ  - 

M=1  N=1  L 


WfN 


1 - I - A/W 

)]  , (63) 


1 - J - B/W 


The  assumptions  made  in  deriving  Equation  (63)  are  that  both  TV  systems 
have  the  same  frame  rate,  same  number  of  TV  lines  per  frame,  saine  aspect 
ratio,  are  sampled  at  the  same  rate,  and  have  linear  optics.  Figure  17 
illustrates  in  block  diagram  form  the  implementation  of  Equation  63. 

The  hardware  implementation  is  presented  and  discussed  in  Chapter  4. 
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If  W Is  not  an  Integer,  the  preprocessing  algorithm  is  more  complex. 
Making  the  same  assumptions  about  the  two  TV  systems  as  made  previously 
and  defining 


U = smallest  integer  < (I  - 1)W  + 1 

V = smallest  integer  - (J  - 1)W  + 1 
X = smallest  integer  - IW 

Y = smallest  integer  - JW 
the  preprocessing  algorithm  becomes: 


1 


I 


y M=U+1  N=V+1 


I HR(M.N) 


Y r 

+ I fu  - (M)wl  HR(u,N)  + [iW  - X ] HR(X+1,N) 
N=v+i  L*- 


X 

+ I 

M=U+1 


["v  - (J-Dwj  HR(M,v)  + pw  -Y  j HR(M,Y+1) 


fu  - (I-l)w]  [v  - (J-l)wj  HR(U,V)  + fjW  - Yj  HR(U,Y+1) 


+ [IW  - X] 


jv  - (0-1  )wj  HR(X+1,V)  + [jW  - yJ  HR(X+1,Y+1) 


Figure  18  illustrates  r,he  pixel  averaging  where  W = 3.25.  The  algorithm  in 
Equation  64  can  be  checi<ed  by  looking  at  the  P(2,2)  element  in  Figure  18. 


(64) 


Figure  18.  Preprocessing  where  W = 3.25. 

The  possibility  that  the  last  column  and  last  row  of  P(I,0)  elements 
turn  out  to  have  less  than  W x W elements  of  the  HR  array  is  of  no  concern 
since  only  the  32  x 32  or  32  x 64  P(I,J)  elements  centered  about  the 
desired  target  will  be  used  in  the  correlation  algorithm.  The  important 
factor  is  to  have  the  spatial  resolution  of  the  preprocessed  HR  array 
identical  to  the  spatial  resolution  of  the  LR  array. 

In  Equation  64,  W is  the  scale  factor  by  which  the  resolution  of  the 
fine  pointing  and  tracking  system  TV  image  must  be  reduced.  The  scale 
factor  is  due  only  to  differing  FOV  and  is  the  same  in  both  the  horizontal 
and  vertical  directions  because  of  the  assumptions  made  about  the  two  TV 
systems.  The  effect  on  Equation  64  of  removing  these  restrictions  will  now 
be  investigated. 

a.  Number  of  TV  lines  per  frame. 

If  the  number  of  TV  lines  in  the  HR  image  is  twice  the  number  of 
lines  in  the  LR  image  and  all  other  factors  are  the  same  including  the 
FOV,  then  the  vertical  resolution  of  the  HR  image  is  twice  the  LR  image 
and  the  horizontal  resolution  is  half.  The  reduction  of  horizonal 
resolution  is  caused  by  the  fact  that  the  image  beam  is  scanning  twice 
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as  fast  In  the  HR  Image  which  causes  each  pixel  width  to  be  twice  as 
long  when  sampling  at  the  same  rate.  This  could  be  corrected  by  sampling 
the  HR  Image  at  twice  the  rate  of  the  LR  Image. 

Let  refer  to  the  horizontal  scale  factor  and  Wy  refer  to  the 
vertical  scale  factor  between  the  HR  and  LR  systems.  Then  If  the  ratio 
of  the  number  of  TV  lines  In  the  LR  to  the  HR  Images  Is  and  all  other 
parameters  are  Identical,  the  contribution  of  the  different  number  of 
lines  to  is  and  to  Wy  is  1/W^. 

b.  Aspect  Ratio 

If  the  LR  IfTiage  has  a 4:3  aspect  ratio  and  the  HR  Image  has  a 1:1 
aspect  ratio  and  all  the  other  parameters  are  identical,  then  the  vertical 
spatial  resolutions  of  the  two  sampled  images  are  the  same  but  the  hori- 
zontal spatial  resolution  of  the  HR  image  is  4/3  that  of  the  LR  image. 

The  reason  for  this  is  that  with  linear  optics  the  LR  horizontal  line  is 
looking  at  a total  scene  which  is  4/3  wider  than  the  HR  scene.  However 
since  the  scan  rate  and  sampling  rates  are  the  same,  then  each  pixel  in 
the  LR  image  covers  4/3  as  much  horizontal  target  c'^ea  as  the  HR  image. 
Therefore  the  horizontal  spatial  resolution  of  the  HR  image  is  4/3 
greater  than  the  LR  image. 

In  general  if  the  aspect  ratios  of  the  LR  and  HR  images  are  P-j  and 
?2  respectively  where  P-f/Pp  ~ ^2’  contribution  of  the  differing 

aspect  ratios  to  is  ^nd  to  Wy  is  1. 

c.  Frame  rate 

If  the  frame  rate  of  the  LR  image  is  30  frames/sec  and  of  the  HR 
image  is  15  frames/sec  and  all  other  parameters  are  equal,  then  the 
horizontal  spatial  resolution  of  the  HR  image  is  twice  that  of  the  LR 
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Imge  and  the  vertical  spatial  resolutions  are  the  same.  This  Is  caused 
by  the  fact  that  the  beam  Is  scanning  twice  as  fast  in  the  LR  system 
thereby  causing  the  spatial  width  of  each  pixel  to  be  double  that  of 
the  HR  Image  when  the  sampling  rates  are  identical. 

In  general.  If  the  ratio  of  the  frame  rates  of  the  LR  to  HR  system 
Is  Wj  and  all  other  parameters  are  identical,  then  the  contribution  of 
the  differing  frame  rates  to  is  and  to  Wy  is  1. 

d.  Sampling  Rate 

By  an  analysis  similiar  to  that  given  in  (a)  - (c)  above,  if  the 
ratio  of  the  sampling  rates  of  the  HR  to  LR  TV  images  is  and  all 
other  parameters  are  identical,  then  the  contribution  of  the  different 
sampling  rates  to  is  and  to  Wy  is  1. 

e.  Field  of  View 

If  the  ratio  of  the  LR  to  HR  fields  of  view  is  and  all  other 
parameters  are  identical,  then  the  contribution  of  the  different 
FOV's  to  W^  is  Wg  and  to  Wy  is  Wg. 

f . General  Spatial  Resolution  Matching  Equation 

If  all  of  the  parameters  in  (a)  through  (e)  above  are  considered 
at  once,  the  total  horizontal  and  vertical  spatial  resolution  scale 
factors  are  given  by 

Wh  = Wj  W^  W3  w^  W5 


Wy  = W^/W^ 


(65) 
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Defining  . 

U « snallest  Integer  - (1-1)^^  ♦ 1 

V » snellest  Integer  - ♦ 1 

X » smallest  Integer  - IWy 

Y « sma'Jlest  Integer  - 

the  preprocessing  algorithm  given  In  Equation  64  can  be  placed  In  the 
more  general  form  given  below. 


P(i.j),A_,  I i 
Wj^Wy  M»UH  N»V^1 


HR(N.N) 


X 

+ I 

N=vn 

Y 

[u  - (l-l)wj  HR(U.N)  + [IH^  - X]  HR(X+l,N)j 
r 1 

* m-Li 

[V  - (J-l)Wj  HR(M,V)  [JH„  - Y]  HR(M,Y+1) 

+ [U-(I-l)Wj  [[V  - (J-l)Mj^]HR(U,V)  + [OWj^  - Y]  HR(U.Y<1)1 

♦ [«,  - 

X]  [CV  - (O-l)Wj^]  HR{X+1,V)  + [JW„  - Y]  HR(X+l,Y+l)jj 

/ 


(67) 

2.  Low  Resolution  Image  Preprocessing 

Since  the  mean  value  used  for  quantizing  each  Kxl.  sub-array  In  the 
LR  Image  Is  different.  It  Is  felt  that  all  of  the  preprocessing  can  best 
be  handled  digitally.  This  section  presents  the  optimal  and  sub-optimal 
techniques  used  to  find  the  mean  value  and  to  quantize  the  pixels  of  each 
KxL  sub-array  for  the  two  level  quantization  case. 
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Optlnally  for  each  value  of  p and  q.  in  Equation  3 and  Hgure  4 the 
K X L pixels  should  be  quantized  about  the  mean  video  level  of  that  sub- 
array. This  could  be  done  by  storing  K lines  of  the  Iom  resolution  video 
in  say  4 or  5 bit  memory,  computing  the  average  value  of  each  K x L 
sub-array  as  each  new  pixel  is  added,  then  quantizing  all  K x L pixels 
about  the  mean  for  each  value  of  p and  q.  It  is  felt  that  this  method 
is  far  too  time  and  power  consuming  for  the  amount  of  accuracy  gained. 

Several  sub-optimal  trade-offs  will  now  be  presented. 

Since  the  mean  video  level  is  not  expected  to  change  appreciably 

from  frame  to  frame,  one  method  of  reducing  the  rea1-tin«  computational 
burden  is  to  use  the  mean  video  level  of  each  sub-array  computed  from 
the  previous  frame.  This  method,  however,  requires  additional  memory 
and  additional  host  computer  load. 

A second  possibility  would  be  to  quantize  about  the  same  mean  for  each 
sub-array  until  the  mean  changes  by  more  than  some  precomputed  amount  at 
which  time  that  entire  K x L sub-array  would  be  requantized  about  the  new 
mean.  This  method,  however,  still  requires  the  hardware  to  requantize  an 
entire  K x L sub^-array. 

A third  possiblity  is  to  store  the  previous  K lines  of  digitized 
video  in  4 or  5 bit  memory,  compute  the  average  value  of  each  K x L 
sub-array  as  a new  pixel  is  digitized,  and  requantized  only  the  last  column 
of  the  array  about  the  new  mean.  This  would  require  only  K comparators 
rather  than  KL  comparators.  The  average  can  be  computed  by  storing  the 
sum  of  each  column  in  shift  registers.  When  a new  pixel  is  added,  the 
first  column  sum  is  subtracted  from  the  total  and  the  column  containing 
the  nev/  pixel  is  added  to  the  total. 
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A fourth  possibility  is  to  use  the  method  of  the  previous  paragraph 

but  quantize  only  the  added  pixel  about  the  new  mean  leaving  all  other 

quantized  values  the  same.  This  would  require  only  one  comparator.  Using 

this  method  only  the  lower  right  pixel  in  the  K x 1.  array  would  be  quantized 

exactly  as  shown  in  the  solid  area  of  Figure  19.  If  the  mean  video  is 
assumed  to  vary  linearly  with  distance  from  this  pixel,  then  the  upoer 

left  pixel  has  the  largest  error  for  this  value  of  p and  q.  The  fifth  and  more 
accurate  method  on  the  average  would  be  to  compute  the  quantization  level 
of  the  marked  pixel  by  computing  the  mean  level  of  the  dashed  K x L array 
as  shown  in  Figure  19.  This  would  mean  a delay  in  each  correlation 
computation  of  approximately  K/2  lines  plus  L/2  sample  intervals. 
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Figure  19.  Method  for  pixel  quantization. 


The  sixth  possibility  is  to  quantize  about  the  average  of  the  HR 
reference.  It  is  felt  that  this  method,  while  being  by  far  the  easiest 
to  implement,  ’..‘ould  lead  to  too  many  false  registration  points  and  would 
be  unacceptable. 

A further  study  involving  typical  military  scenes  is  needed  to 
determine  which  of  the  above  trade-offs  is  the  best  to  implement.  Without 
the  benefit  of  this  study,  method  five  is  recommended  and  an  implementation 


s suggested  in  Chapter  4 of  this  report. 
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3.  Effect  of  Quantization  about  Incorrect  Mean 


As  pointed  out  in  the  previous  section,  one  of  the  sources  of  error 
in  the  two  level  by  two  level  (one-bit  by  one-bit)  correlator  is  quanti- 
zation about  an  incorrect  ir..an.  The  effect  on  the  mean  square  signal/ 
noise  ratio  of  quantizing  an  entire  array  about  the  same  incorrect  mean 
is  presented  in  this  section.  The  effect  on  the  ms  S/N  ratio  of 
quantizing  each  pixel  about  differing  incorrect  means  is  not  presented. 
However,  the  results  can  be  used  as  an  upper  bound  if  one  uses  the 
maximum  expected  mean  error  in  the  resulting  curves. 

Consider  the  HR  and  LR  images  quantized  respectively  with  the 
quantizers  shown  in  Figure  20.  These  quantizers  represent  an  error  in 
quantizing  around  the  wrong  dc  level  of  the  HR  and  LR  images.  Although 
the  above  quantizers  assume  that  the  HR  and  LR  signals  are  zero  mean, 
this  does  not  necessarily  have  to  be  true.  For  HR  and  LR  signals  that 
are  not  zero  mean  the  dc  error  due  to  quantization  would  be  given  by 
aX^  = X^-HR  and  aX^  = X2-LR. 


Figure  20.  Quantizers  for  the  HR  and  LR  images. 
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a.  Direct  Method 

Con’iider  the  first  line  of  video 


R(1.q)  = g l_  HRd.m)  LR0,m+q) 


(68) 


m=l 


Dropping  the  row  1 notation  and  quantizing  the  HR  and  LR  signals  as  shown 
in  Figure  20  yields 

L 


R(q)  = ^ ^ (RR(”»)I  ^2  (^^("*+^3)1 

m=l 


The  variance  in  the  estimate  of  R is 


O/s  = <R  > - <R> 


R 


where 


<R> 


■■lU.’* 


[HR(m)l  g2[LR(m+q)] 


and 


<R  > = E 


1 ^ 

r I g, [HR(ra)j  g [LR{m+q)] 
^ m=l  ^ 


(69) 


(70) 


(71) 


1 2 


(72) 


Assuming  sample  pair  independence,  a gausian  distribution  of  the 
HR  and  LR  signals,  and  evaluating  the  resulting  expectation  integrals 
vields 


0.^  = i I 1 - 16  erf^  (^)  erf^  (^)] 


(73) 
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Using  Price's  Theorem  to  evaluate  r(t)  where  p is  the  correlation 
coefficient  between  HR  and  LR  yields 


dp 


2 /^1. 


)> 


(74) 


Evaluating  the  partials  in  Equation  74  by  inspection  of  Figure  20, 


assuming  a jointly  gaussian  distribution  for  the  HR  and  LR  signals  and 

2 2 

evaluating  yields  the  following  relationship  between  0 and  0.  . 

P R 

V 2 2 

^ "2 


(75) 


^ 2 _ ,71,2  a"  ^ ^ 2 

o = (2)  e e 

^ R 


Substitution  of  Equation  73  into  75  yields  the  desired  effect  of  the 
incorrect  mean  on  the  ms  S/N  ratio. 


^1  ^ ^2 

0 ^ 

L T"  ® ® 


2 1 tt2  ^ (0— ) 


X2  2 


16erf^(-‘^:)  erf^^-l)| 


(76) 


b.  SSDA  Method 


The  registration  function  for  the  SSDA  method  is  given  by 


1 ^ 

^(^)  = L I |HR(m)  - LR(m+q) I (77) 

m=l 

Going  thru  an  analysis  similiar  to  that  used  for  the  Direct  Method 

2 

yields  the  identical  result  for  0^^  given  in  Equation  76.  This  was 
expected,  however,  since  the  two  methods  have  been  shown  to  be  equiva- 
lent in  the  one-bit  by  one-bit  case. 
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Figure  21  shows  a parametric  set  of  curves  depicting  the  change  in 
0 ^ for  various  values  of  (^)  and  (— ).  The  dotted  curve  in  Figure  21 

P a a 

2 

shows  the  effect  on  if  the  dc  errors  in  the  quantizers  are  equal. 

If  both  and  X2  are  zero  (i.e. , no  error  in  the  mean),  then 
2 

Op  X L = 2.467  which  is  the  result  obtained  in  Table  1.  As  can  be  seen 
from  Figure  21  the  variance  is  essentially  insensitive  to  small  dc 
errors  but  grows  very  rapidly  for  errors  in  the  mean  above  0.4a. 


Figure  21.  Effect  of  dc  quantization  level  on  the  variance  in  the 
estimate  of  p 
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4.  Preprocessing  for  Differing  Sensor  Characteristics 

When  trying  to  accomplish  target  hand-off  between  two  systems,  it 
is  very  important  for  the  two  sensors  to  have  similiar  characteristics. 
This  of  course  is  the  reason  for  the  necessity  for  the  algorithm  which 
equalizes  the  spatial  resolution  of  the  two  images  as  given  in  Section 
C-1  of  this  chapter.  It  is  also  very  important  for  the  spectral  charac- 
teristics of  the  two  sensors  to  be  the  same  since  correlation  can  be 
accomplished  only  if  the  two  video  signals  are  almost  the  same.  If  one 
sensor  covers  a wider  frequency  spectrum  than  the  other,  an  optical 
filter  might  possibly  be  used  during  correlation  to  cause  the  two 
spectral  responses  to  be  the  same.  If  the  two  sensors  have  completely 
different  spectral  responses,  such  as  one  sensitive  in  the  visible 
spectrum  and  one  in  the  infrared  spectrum,  then  correlation  would  be 
difficult  to  accomplish  due  to  the  difference  in  signal  characteristics. 
However  one  possibility  in  this  case  would  be  to  preprocess  the  video 
such  that  only  the  edges  are  used  for  correlation.  This  is  a possibility 
since  even  if  there  is  a contrast  reversal  between  the  two  signals,  the 
edges  in  the  FOV  should  be  approximately  the  same. 

Preprocessing  of  the  HR  and  LR  arrays  concerns  changing  the  form 
of  the  data  so  that  a better  registration  between  the  two  can  be 
obtained.  In  most  of  the  literature  that  has  been  surveyed  this  task 
is  referred  to  as  outlining,  edge  detection,  or  boundary  mapping.  These 
methods  are  used  mainly  for  cases  in  which  the  two  images  are  acquired 
at  different  times  jr  the  images  come  from  different  types  of  sensors. 

The  first  of  these  reasons  is  not  applicable  to  the  present  system  under 
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study.  It  Is  quite  possible  that  for  the  system  under  study  the  Images 
could  be  produced  from  different  types  of  sensors.  In  this  situation 
the  data  representation  of  the  ground  scene  from  the  two  sensors  might 
be  different.  Than  some  type  of  preprocessing  of  the  data,  such  as 
boundary  mapping  could  be  employed. 

Reference  [5]  describes  a method  by  which  boundary  maps  are  pro> 
duced  from  the  raw  data.  Essentially  this  method  calculates  the 
differential  change  In  magnitude  from  one  data  point  to  the  next.  It 
does  this  for  both  X and  Y directions  away  from  the  point  being  examined. 
The  algorithm  then  decides  if  the  point  being  examined  represents  the 
boundary  of  a region  or  not.  If  It  is  a boundary,  then  that  location  is 
assigned  a value  of  1.  If  it  is  not  a boundary  point,  a 0 is  assigned 
to  the  location.  Once  a boundary  map  is  produced  for  the  entire  image, 
a correlation  can  be  performed  using  boundaries  of  objects  in  the  image 
instead  of  all  the  raw  data  from  the  image. 

If  the  data  received  from  the  HR  and  LR  imaging  systems  are  of 
different  forms  (such  as  would  occur  with  different  sensor  characteris- 
tics) then  the  method  of  boundary  maps  or  edge  detection  would  warrant 
further  investigation.  Implementation  of  any  of  these  methods,  however, 
requires  additional  hardware  and  computation  time. 

D.  Vibration  Analysis 

The  effect  of  LOS  jitter  of  the  PTS  and  missile  seeker  systems  on 
the  correlation  algorithm  accuracy  is  investigated  in  this  section.  From 
talks  with  MICOM  engineers  the  jitter  of  the  seeker  LOS  should  be  less 
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than  100  urad  rms  and  Is  the  value  used  in  this  section.  The  LOS  jitter 
of  the  PTS  system  has  been  demonstrated  to  be  less  than  30  yrad  for  at 
least  one  system.  These  values  are  the  total  rms  jitter  due  primarily 
to  disturbances  from  the  helicopter  at  11,  22,  33,  and  44  Hz. 

The  primary  effect  of  LOS  jitter  on  the  correlation  algorithm  is 
to  cause  a distortion  in  the  stoi*ed  video  scene.  For  example,  if  the 
jitter  is  causing  the  LOS  to  move  downward  during  the  sampling  and 
storage  of  a 32  x 64  element  video  array,  then  the  32nd  line  may  actually 
be  looking  at  the  scene  the  equivalent  of  several  TV  lines  below  where 
it  should  be  thus  causing  a stretching  distortion.  If  the  LOS  is  moving 
up  due  to  jitter,  then  the  32nd  line  may  actually  be  looking  at  the 
scene  the  equivalent  of  several  TV  lines  above  where  it  should  be  thus 
causing  a compressing  distortion.  The  problem  is  further  complicated 
by  the  fact  that  one  video  image  may  be  compressed  and  the  other  stretched 
by  the  jitter.  The  net  effect  on  the  correlation  algorithm  is  to  cause 
a smaller  correlation  peak  which  can  be  modeled  by  a reduction  in  the 
sianal-to-noise  ratio 

In  the  calculations  which  follow  both  TV  systems  are  considered  to 
be  525  line  systems  with  480  active  TV  lines  (the  remaining  45  lines  are 
used  in  the  vertical  retrace),  and  to  have  60  half-frames  per  second  or 
30  full  frames  per  second  (2:1  interlace)  with  a 4/3  aspect  ratio.  For 
a sampling  rate  of  7.5  MHz  the  low  resolution  image  will  be  digitized 
into  a 240  x 480  element  array  during  the  first  half-frame  and  240  x 480 
element  array  during  the  second  half- frame.  If  the  correlator  is  designed 
such  that  correlation  values  are  calculated  as  data  comes  in,  then  correla- 
tion is  being  performed  during  each  half-frame  (i.e.,  the  32  rows  used  in  the 
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correlator  all  cone  fron  the  sane  ha1f*frane).  Ther«fore»  the  total 
tine  elapsed  fron  the  first  sanple  In  each  32  x 64  array  used  In  the 
correlation  algorlthn  to  the  last  will  be  the  tine  required  to  sanple 
31  conplete  lines  plus  64  sanples  of  the  following  line. 

T , 31  « 0.00197  seconds  (78) 

^ 525  X 30 

As  stated  previously  the  LOS  jitter  Is  predoarinately  at  11,  22, 
33,  and  44  Hz.  If  one  assunes  that  the  100  yrad  ms  Jitter  Is  all  at 
44  Hz,  the  worst  possible  condition  for  correlation,  then  the  greatest 
angular  disturbance  which  the  LOS  would  have  In  0.00197  seconds  would 
be  twice  the  value  of  the  44  Hz  sinewave  evaluated  at  t - .00197/2 
seconds. 


= 2041.4)  si n[2T:44 (0.000986)] 

IiIqX 

or 

If  the  seeker  vertical  FOV  is  2®,  then  each  TV  line  is  scanning  72.7 
yrad  if  one  assumes  480  active  TV  lines.  Therefore  the  maximum  movement 
of  the  LOS  from  the  first  to  the  last  sample  taken  for  each  computation 
of  R would  be  slightly  over  one  line.  Since  correlation  in  the  half- 
frame correlator  is  being  performed  by  using  alternate  lines  of  the 
full  frame,  it  is  felt  that  this  amount  of  movement  will  not  cause  a 
significant  problem.  A more  detailed  analysis  could  be  obtained  by 
modeling  this  LOS  movement  with  an  additive  noise  source  which  would 
degrade  the  S/N  ratio. 
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Next  consider  the  case  where  correlation  Is  perforaed  using  16  lines 
froa  one  ha1f>fraae  and  16  lines  froa  the  succeeding  half-fraae.  The 
32  X 64  correlator  then  operates  on  64  saaples  froa  each  of  these  lines. 
The  total  time  required  to  saaple  the  data  used  for  each  coaputatlon  of 
R Is 

T-  • IS  * ♦ _L  . 0.0176  seconds  (80) 

525  - 30  60 

which  Is  aore  than  half  a cycle  of  a 44  H2  wavefora.  Therefore,  the 
total  aoveaerjt  could  be  as  auch  as  the  peak-to-peak  LOS  jitter  or  282 
wrad.  This  represents  approximately  3.89  TV  lines.  Even  If  all  of  the 
jitter  Is  at  11  Hz,  the  aaxiaua  angular  moveaent  would  be  162  prad  which 
Is  over  2 TV  lines.  Using  the  full-fraae  correlator  the  correlation 
peak  would  be  reduced  more  than  for  the  half-frame  correlator.  The 
net  effect  of  the  LOS  jitter  is  a larger  reduction  in  the  S/N  ratio  for 
a full-frame  correlator  than  for  a half-frame  correlator. 

Table  3 shows  the  ri^ximum  possible  angular  vibration  in  prad  and 
Its  equivalent  in  TV  lines  if  a 100  urad  rms  jitter  in  the  seeker  LOS 
is  considered  to  be  concentrated  at  11,  22,  33,  or  44  Hz.  The  data  for 
a 32  X 64  element  correlator  operating  on  each  half-frame  respectively 
and  for  the  same  correlator  operating  on  a full-frame  are  given.  ^j22L 
the  data  in  Table  3 it  is  obvious  that  a 32  x 64  correlator  operating 
on  successivie  half-frames  is  much  less  sensitive  to  LOS  jitter  than  a 
full-frame  correlator.  The  half-frame  correlator  is  also  simpler  to 
lement  since  data  does  not  have  to  be  stored  from  one  half-frame  to 
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Frequency 

Half -Frame 
Correlator 

Full-Frame 

Correlator 

Hz 

A<^ 

max 
U rad 

Equivalent 
TV  Lines 

^max 
p rad 

Equivalent 
TV  Lines 

11 

20 

0.27 

162 

2.22 

22 

40 

0.54 

265 

3.64 

33 

59 

0.81 

282 

3.86 

44 

78 

1.07 

282 

3.86 

Table  3.  Angular  disturbances  in  seeker  LOS  due  to  jitter. 


Next  consider  a PTS  system  with  a vertical  FOV  of  0.4®.  With  this 
system  five  lines  of  video  must  be  averaged  in  order  to  equalize  the 
size  of  the  target  in  the  two  FOVs.  Therefore  the  elapsed  time  between 
first  and  last  sample  will  be 


T_  » (5  X 32«1 ) > (64/4601  - q q]  seconds 
^ 525  + 30 


(81) 


If  one  assumes  that  the  30  urad  of  jitter  is  all  at  44  Hz,  then  the 
largest  angular  disturbance  which  the  LOS  would  have  in  .01  seconds 
would  be 


or 


= 2{^)  (30)sin[2ir44(.005)] 
max 


A(t  = 83.35  virad 
^max 


(82) 


{ 


For  a vertical  FOV  of  0.4®,  each  TV  line,  assuim'nq  480  active  lines, 
scans  14.5  yrad.  Since  each  five  successive  lines  of  the  PTS  video  are 
averaged,  the  is  slightly  more  than  one  TV  line  when  averaged. 

The  net  effect  of  LOS  jitter  on  the  PTS  is  therefore  seen  to  be 
approximately  the  same  as  the  effect  on  the  seeker  system. 

E.  Derivation  of  aX  and  AY  Misalignment  in  LR  Image 
After  the  correlation  array,  R(p,q),  has  been  calculated,  it  is 
desired  to  move  the  missile  seeker  LOS  such  that  the  target  is  in  the 
center  of  the  LR  FOV.  In  order  to  do  this  the  misalignment  in  both  the 
X and  y directions  must  be  determined.  The  position  of  the  target  in 
the  LR  image  can  be  determined  by  performing  a search  of  the  R(p,q) 
correlation  array  for  a maximum  in  order  to  identify  the  registration 
points  p*  and  q*. 

The  geometrical  layout  of  the  HR  and  LR  images  is  shown  in  Figure 
22.  The  p and  q counters  start  in  the  upper  left  corner  of  the  LR 
image.  This  designates  the  starting  placement  of  the  HR  image  in  the 
LR  image  for  the  calculation  of  the  R(p,q)  array. 

Figure  22  also  shows  a possible  location  of  the  HR  Image  which 
produced  a maximum  value  of  R(p,q)  over  all  possible  p and  q.  It  is 
desired  that  image  registration  take  place  in  the  center  of  the  LR 
image.  The  K x L array  of  LR  image  points  around  the  center  is  shown 
in  Figure  23  along  with  the  geometric  definitions  of  aX  and  aY.  The 
displacements  AX  and  aY  represent  values  by  which  the  LR  camera  must  be 
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Figure  22,  Image  layout  identifying  p*  and  q*. 


X 


?8 


shifted  so  that  the  maximum  value  of  R(p,q)  will  occur  at  the  center  of 
the  LR  image.  It  is  desired  that 

p*  = N--.i  (83) 

2 

and 

q*  = 5-^  , (84) 

Using  this  reasoning  aX  and  aY  can  be  calculated  using  the  following 
equations: 

AX  = q*  - {^)  (85) 

and 

aY  = (M)  - p*.  (86) 

2 

The  AX  and  AY  in  Equations  85  and  86  are  expressed  in  terms  of  pixel 
elements.  The  angular  errors  A4ij^  and  in  a seeker  with  a FOV  of  2^, 
480  active  TV  lines  in  this  FOV  and  480  samples  per  line  are 

in  degrees  (87) 

in  degrees  (88) 

These  angular  errors  can  be  used  to  torque  the  seeker  gimbals  such 
that  the  target  is  in  the  center  of  the  FOV. 


A(j)y  = ^ X 2°, 
I Aon 
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F.  Threshold  Effects  on  Probability  of  False 
Registration  and  Detection  Probability 

The  following  analysis  develops  the  concepts  of  the  probability  of 
false  registration  (i.e. , the  probability  that  two  images  are  chosen  as 
registered  when  they  in  fact  are  not)  and  the  probability  o^  detection 
(i.e.,  the  probability  that  two  images  are  chosen  to  be  registered 
given  that  they  are  in  fact  registered).  Each  of  these  probabilities 
is  given  in  terms  of  the  threshold  for  deciding  if  registration  is 
achieved  and  the  input  SNR  for  both  images. 

For  simplicity  and  because  other  considerations  indicate  its 
desirability,  the  following  is  developed  fcr  a 1-bit  by  1-bit  (2-level 
by  2-level ) correlator  using  either  the  Direct  Double  Summation  Method 
or  the  Sequential  Similarity  Detection  Algorithm.  Previous  comparisons 
have  shown  these  two  methods  to  be  identical  statistically  for  the  1-bit 
quantization  case. 

The  following  assumptions  are  inherent  in  the  devp’opment. 

1.  Signal  and  noise  levels  at  each  pixel  are  independent  of  each 
other  and  of  signal  and  noise  levels  from  any  other  pixel. 

2.  The  noise  at  any  pixel  is  zero  mean  Gaussian  for  both  the  HR 
and  LR  images. 

3.  The  rms  noise  level  is  constant  (o  ) for  each  pixel  of  the 

*^HR 

HR  imace  and  a different  constant  (a  ) for  each  pixel  of  the  LR  image. 

"lr 

4.  The  signal  levels  for  the  HR  and  LR  images  have  mean  values 

HR  and  LR  respectively  for  the  pixels  within  each  image.  It  is  of  course 
realized  that  LR  may  change  as  p a.id  q (the  shift  values)  change;  however 
for  a gi''en  correlation  computation  Ol  is  a constant. 
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5.  The  signal  plus  auditive  noise  from  HR  and  LR  images  are 
subjected  to  1-bit  quantization  as  represented  by  g-j  and  §2  in  Figures 
24  and  25. 

The  only  possible  outputs  for  a one-bit  implementation  are  zero  or 
one  for  comparison  of  a LR  pixel  with  a HR  pixel.  Therefore,  for 
implementation  of  a K by  1.  array  correlation,  the  correlation  magnitude 
takes  on  values  ranging  from  0 to  KL  in  integer  steps.  The  ouvious 
questions  which  arise  and  the  answer  of  which  is  the  purpose  of  this 
section  is:  "How  close  to  the  maximum  value  of  KL  must  a given  corre- 

lation be  to  indicate  registration?  What  are  the  consequences  of 
chosing  a particular  threshold  for  a decision  on  registration  of  the 
HR  and  LR  images?" 

Under  the  assumptions  outlined  above  we  can  write  the  probability 
density  functions  for  the  noise  in  any  pixel  of  the  HR  and  LR  images 
for  a given  shift  value  (p,q). 


-n 


HR 


2a. 


'HR 


\/?rfO 

^ "hr 


(89) 


-n 


LR 


2a. 


' R 


'LR 


"lr 


(90) 


At  pixel  (n,m)  in  the  HR  image  and  pixel  (n+p,  ni+q)  in  the  LR  image  we 
have  signal  plus  noise  as  follows: 


^HR  " " HR(n,m)  + n^^(n,m) 


(91) 


r^R  = rj^R(n+p,  nH-q)  = LR(n+p,nH-q)  + n^^R(n+p,  m+q) 


( 


where  r^R  and  r^R  are  to  be  quantized  and  compared  logically  and  the 
output  summed  for  KL  pixels  as  shown  in  Figure  26. 


Figure  26.  Correlation  technique. 


From  the  quantizers  g-j  and  92  we  have  the  rules  for  no  noise 


j 1 if  HR  > HR 
(0  if  HR  < HR 


(93) 


92  = (1  if  LR  > LR 
= (0  if  LR  < LR 


(94) 


If  noise  is  included,  the  rules  become  with  noise 

> 

H if  HR+n^P  > HR 

= H if  ^ 


(95: 


92 


1 if 
0 if 


LR+Olr  > 


LR 

IR 


(96 
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Clearly  there  are  four  hypotheses  that  can  be  postulated  for  the 
outputs  of  the  two  quantizers  in  terms  of  the  inputs.  The 
hypotheses  are: 

Hhro  - the  input  HR  < HR  (choose  g-|=0) 

Hrr^  - the  input  HR  > HR  (choose  g^=l) 

Hlro  - the  input  LR  < LR  (choose  g2=0) 

Hlri  - the  i nput  LR  > LR  (choose  g2'l ) 

These  hypotheses  can  be  chosen  with  no  error  by  a simple  threshold 
circuit  if  no  noise  is  present.  If  noise  is  present,  there  are  possi- 
bilities for  error  in  the  choices.  This  error  is  dependent  on  the  signal 
levels,  the  noise  statistics  and  the  number  of  pixels  used  in  a given 
correlation  computation. 

The  correlation  output  is  then  compared  with  a threshold  value,  Nq, 
to  determine  if  the  images  are  registered.  If  the  correlation  output 
is  greater  than  Nq,  the  images  are  chosen  to  be  registered.  Since  Nq 
is  a parameter  which  must  be  chosen,  its  effect  on  expected  errors  must 
be  known.  Specifically  its  effect  on  false  alarm  and  detection  proba- 
bilities must  be  known. 

1 False  Alarm 

False  alarm  occurs  when  the  logic  comparison  indicates  registration 
of  the  image  with  a pa»’tirular  LR  image  when  in  fact  the  two  are  different. 
The  probability  of  false  alarm  is  best  examined  by  considering  the 
probability  of  false  alarm  for  each  comparison  of  an  HR  and  an  LR  pixel 

i 
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pair.  Defined  for  each  pixel  pair,  the  false  alarm  probability  is 
simply  the  probability  that  the  HR  and  LR  pixels  are  chosen  matched 
given  that  they  are  different.  However,  since  the  HR  and  LR  images 
can  be  different  (not  registered)  without  requiring  that  all  KL  pixels 
be  different  between  the  two  images,  consideration  must  be  given  for 
the  probability  of  matched  pixel  pairs  within  non-recistered  images. 

This  consideration  requires  an  additional  assumption  about  the  proba- 
bilities of  pixel  pairs  being  matched  or  mis-matched  given  that  the 
images  are  different.  Fortunately,  based  on  fundamental  arguments 
supported  by  the  method  of  quantizing  about  the  mean  value  for  each 
image,  these  assumed  probabilities  are  easily  obtained. 

Since  the  relation  of  total  number  of  chosen  pixel  matches  to  the 
threshold  Ng  is  really  what  determines  false  registration  probability, 
it  is  necessary  to  examine  the  probability  of  citoosing  pixels  as 
matched  given  that  the  images  are  different.  Clearly  this  has  two 
parts:  1.  probability  of  [choosing  pixels  matched|pixels  matched| 

images  different],  and  2.  probability  of  [choosing  pixels  matched| 
pixels  different  I images  different]. 

If  the  individual  pixels  for  different  images  are  assumed  to  be 
independent  of  each  other,  then  a "0"  or  "1"  is  equally  likely  at  any 
given  quantization  step.  This  should  be  true  for  any  symmetric  proba- 
bility density  functions  of  the  HR  and  LR  intensities  since  quantization 
is  about  the  mean  value  in  each  image.  With  this  assumption,  it  can  be 
shown  that  for  each  pixel  pair  comparicon,  the  probability  of  false 
alarm  is  independent  of  the  image  signal-to-.soise  ratios. 
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The  probability  of  false  alann  for  the  ith  pixel  pair  comparison 


P,[E]»tH„,MnMl-HHR)0-ML,)]  (98) 

•P[pixels  match  1 di fferent  images] 

+ M^p(l-Mj^p)]*P[pixels  different! images  different] 


where; 


SNRHR 


”hr  = 


1-  e ^ de 


SNRLR 


— e ^ dB 


^J27 


SNRHR  = HR  image  signal-to-noise  ratio 
SNRLR  = LR  image  signal-to-noise  ratio. 


Under  the  assumption  stated  above  P[pixels  match jimages  different] 
= P[pixels  different! images  different]  = and  P^.[E]  = for  all  SNRHR 
and  SNRLR. 

If  all  KL  pixels  are  considered,  we  find  that  if  N is  the  number 
of  false  alarms  in  KL  pixels  then  the  probability  density  function  for 


N is  given  by 


■W 


PM[E]  = rUa'^b 


.N^KL-N) 


where  Pf^[E]  is  the  probability  of  exactly  N false  alarms  and  obeys  the 
familiar  binomial  probability  law 
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where 


(KL)» 

N!(KL-N)! 


(100) 


is  the  binomial  coefficient  and  gives  the  number  of  possible  combinations 
of  KL  pixels  with  N false  alarms,  and 


a = P^[E]  = probability  of  false  alarm  on  each  pixel  pair 
b = 1 - P.[E]. 


If  a threshold  value  of  N is  chosen,  i.e.  N = Nq,  such  that  if 
N £ Nq  for  any  given  correlation  computation  the  decision  is  to  reject 
the  hypothesis  that  the  images  are  alike,  then  a probability  of  false 
registration  can  be  found  as  a function  of  Nq  from 


or 


= Probability  [N  > Nq| images  different] 


p„[No:  = 1 


/ (KL)  aNi<KL-N) 
N=0  \ '^/ 


(101, 


For  KL  large,  the  binomial  probability  law  is  approximately  gaussian 
which  gives 


P„(No>  =.  1 - PI 


where 


M(x)  = 


X 

I i 


^Nq  - (Kl.)a  + ^ 
(KL)ab 


e ^ de. 


(102) 


^[27 


I 

I 
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Probability  of  false  alarm  is  plotted  versus  threshold,  Nq,  for 
KL  = 2048  in  Figure  27  and  for  KL  = 1024  in  Figure  28. 

2.  Detection 

Detection  or  correct  registration  occurs  when  the  HR  and  LR  images 
are  correctly  chosen  to  be  identical;  that  is,  they  are  chosen  matched 
given  that  they  are  matched.  Similar  to  the  development  for  false 
alarm,  it  is  convenient  to  start  with  a single  pixel  pair  comparison  to 
derive  the  statistical  behavior  of  correct  registration. 

The  major  difference  between  the  probabilities  for  false  alarm  and 
detection  lies  in  the  fact  that  when  the  HR  and  LR  images  are  registered, 
the  probability  that  pixel  pairs  match  should  be  higher  than  it  was  for 
the  unlike  images  examined  in  the  false  alarm  treatment.  Furthermore, 
this  difference  in  probability  of  match  and  mismatch  for  each  pixel 
pair  results  in  the  probability  being  dependent  on  signal-to-noise 
ratio's  of  the  two  image  signals. 

The  probability  of  choosing  a pixel  pair  matched  given  that  the 
HR  and  LR  images  are  registered  is 

P^[C]  = + (1  - - M^r)] 

•P[pixe1s  match|images  same] 

+[(1  - M^r)M|_r  + 


4 

'■3 


•P[pixels  mismatch  I images  same] 


(103) 


qure  27.  Detection  anH  f;i1se  alarm  probabilities  versus  decision  threshold  (KL=2048) 


0.9999999 


Detection  and  false  alarm  probabilities  versus  decision  threshold  (KL*1024) 
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or  If  we  define  the  error  probability 

e 4 P[p1xe1s  mismatch  I Images  same] 

then 


P,[C]  . * (1  - - H^,)]  (l-c) 

+[(1  - + H^(l  - c . (,(„) 

For  the  Ideal  case,  e = 0,  each  HR  and  LR  pixel  pair  will  match  so 
that  additive  noise  is  the  only  error  source.  However,  for  practical 
sensors,  there  is  a finite  probability  that  pixel  pairs  will  mismatch 
due  to  different  spectral  responses,  non-linearities  and  other  error 
sources.  Thus  it  is  important  to  examine  the  effects  of  non-zero 
values  for  e.  Since  ana  are  dependent  on  signal-to-noise  ratios 
SNRHR  and  SNRLR  of  the  respective  video  signals,  then  as  indicated  by 
Equation  104  the  probability  of  choosing  a single  pixel  pair  as  being 
matched,  given  that  the  two  images  are  alike,  is  dependent  on  the  three 
parameters  SNRHR,  SNRLR  and  e. 

Once  the  value  for  P^[C]  is  established  then  the  total  detection 
probability  as  a function  of  the  threshold,  Nq,  can  be  determined.  If 
all  KL  pixels  are  considered,  we  find  as  with  the  false  alarm  problem 
that  if  N is  the  number  of  pixels  chosen  as  matched,  then  the  probability 
density  function  for  N is  again  binomial. 

P^tC]  = ffc)  d(^L-N) 


(105) 
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And  the  probability  of  detection  Is  the  probability  that  N will  be 
greater  than  Nq,  or 

Pq[Nq]  = Probability  [N  > Nq| Images  alike] 

= ' - ? f!i>"  <’«) 

where 


c = P^[C] 
d = 1 - P.[C] 

Clearly,  is  a function  of  KL,  Ng  and  P^.[C]  which  In  turn  depends  on 
SNRHR,  SNRLR  and  e.  Figure  27  shows  Pg  versus  threshold,  Ng,  for 
KL  = 2048  data  points  and  for  three  worst  case  values  of  P^[C].  (NOTE: 
P^[C]  is  listed  as  PIC  on  all  figures). 

A value  of  PIC  = 0.5  gives  exactly  the  same  curve  for  Pg  as  is 
shown  for  Pp^.  This  could  occur  only  If  we  assume  a value  of  e = 0.5. 
This  is  highly  unlikely  since  it  says,  that  even  thougn  the  images  are 
alike,  there  is  at  best  a 50-50  random  chance  that  individual  pixel 
pairs  match  between  the  two  images.  If  the  sensor  characteristics  are 
so  different  that  e < 0.5  is  possible,  then  there  is  no  need  to  waste 
effort  trying  to  correlate  the  two  images.  Figure  28  shows  similar 
curves  for  Pg  with  KL  = 1024. 

As  an  example  of  what  the  curves  in  Figures  27  and  28  mean  consider 
that  it  is  desired  that  a high  Pg  be  attained  while  keeping  the  number 
of  false  detections  to  one  per  second.  The  number  of  false  detections 
is  given  by 
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where: 

N = number  of  shift  values  "p"  for  which  correlation 
^ is  computed 

Nq  = number  of  shift  values  "q"  in  the  other  direction 

F = frame  rate  (frames/ sec) 

= total  mission  time  (1  sec) 
m 


Npp  = Pp^  • [640-64][480-32]  * 30 

(107) 

1 - 8(10®)Pp^ 

which  requires  Pp^  < 1.2(10"^).  Then  from  Figure  27  this  Pp^  corresponds 
to  a threshold  of  Nq  = 1140. 

Projecting  up  to  the  Pp  curve  for  PIC  = 0.55  indicates  a detection 
probability  of  less  than  0.3  which  is  unacceptable.  For  PIC  = O.G  the 
value  of  Pp  is  greater  that.  0.9999.  Evaluation  of  this  result  says 
that  if  a false  alarm  rate  of  one  per  second  is  to  be  achieved  then  to 
maintain  high  detection  probability,  SNRHR,  SNRLR  and  s:  must  be  such 


that  PIC  > 0.55  and  preferably  ^ 0.6. 

The  relationship  between  Pp  and  Pp^  is  shown  more  clearly  in  Figure 
29  where  Nq  is  the  common  variable  which  has  been  removed.  Improvement 
in  performance  is  represented  by  movement  of  the  curves  diagonally 


upward  from  the  coordinate  center. 


Incidentally,  a false  alarm  rate  of  one  per  second  is  the  case 
where  one  false  registration  occurs  out  of  “SilO^)  correlations. 

This  is  the  number  of  real  time  correlations  computed  per  second  for  a 


Piqure  29.  Probability  of  detection  versus  probability  of  false  alarm 
"Parameter  dependence. 
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32  X 64  reference  arrey  and  a 480  x 640  Iom  resolution  image  at  standard 
TV  frame  rates. 

Sim:e  the  above  results  are  so  strongly  dependent  on  PIC  a parametric 
analysis  of  PIC  is  of  value. 

The  effect  of  e on  the  value  of  PIC  is  shown  in  Figure  30  for  four 
discrete  values  of  e as  a function  of  signa1-to-noise  ratio.  For 
convenience  in  plotting,  only  values  corresponding  to  equal  SNR  for  the 
video  signals  are  shown.  The  figure  shows  that  e has  a profound  effect 
on  the  maximum  attainable  value  of  PIC  which  is  reached  asymptotically 
as  the  signal >to>noise  ratios  increase.  In  each  case,  the  maxitmim 
achievable  value  of  PIC  is  l>c,  and  is  achieved  only  as  the  two  video 
SNR's  increase. 

Since  the  HR  image  SNR  and  the  LR  image  SNR  will  most  likely  vary 
Independently  of  each  other,  it  is  of  interest  to  examine  the  effect  of 
different  SNR's  on  PIC.  Figure  31  illustrates  this  effect  for  a fixed 
value  of  e - 0.1.  As  in  Figure  30  the  absolute  maximum  attainable  PIC 
is  1-e;  however,  it  is  not  reached  unless  both  video  SNR's  are  greater 
than  3.  The  point  made  by  Figure  31  is  that  for  optimum  detection 
probability,  both  images  should  have  a SNR  of  at  least  three.  Only 
partial  compensation  is  achieved  by  one  video  signal  having  very  high 
SNR  If  the  other  video  signal  has  SNR  significantly  less  than  3, 

To  summarize  briefly,  the  above  analysis  Indicates  that  for 
s1gnal-to»noise  ratios  which  can  reasonably  be  expected  from  electro- 
optic  sensors  ('3),  even  for  a worst  case  assumption  about  sensor 
characteristic  mismatch  (e  = 0.4),  for  the  32  x 64  reference  array  size 


SNRLR 

Figure  31.  Effect  of  different  video  SNR  on  probability  of 
making  correct  decision  for  each  pixel. 
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a threshold  can  be  set  »>hich  will  ensure  low  false  alarm  rate  and  high 
detectlop  probability.  It  should  be  stated  that  the  above  analysis  is 
based  on  a one^look  correlation  for  each  shift  value.  Hul tipi  e-look 
correlation,  that  is,  computing  the  correlation  function  for  a given 
shift  value  several  times  and  using  a decision  based  on  the  multiple 
correlations,  will  provide  even  better  performance  statistically.  How- 
i ever,  since  nwltlple  correlations  in  real  time  would  require  parallel 

f 

processors,  increased  processing  speed  or  reduced  data  array  size,  that 
technique  can  be  eliminated  for  practical  reasons.  More  than  adequate 
statistical  performance  should  be  achievable  with  the  one-look  real-time 
correlation  technique. 


r 


G.  Target  Not  in  Seeker  FOV 

Due  to  inaccurate  bores ight  alignment  or  to  movement  of  the  missile 
within  its  stores  rack,  there  exists  the  possibility  that  the  target 
will  not  be  within  the  seeker  FOV.  If  the  correlation  coefficient  does 
not  exceed  the  threshold  value  during  n consecutive  TV  fields,  the 
decision  is  made  to  slew  the  seeker  TV  LOS  in  a search  pattern  to  find 
the  target.  The  value  of  n is  a design  parameter  which  should  be 
studied  in  a technology  program.  A value  of  from  one  to  four  would  be 
an  adequate  range  for  such  a study. 

Figure  32  illustrates  two  possible  search  patterns.  Figure  32(a) 
shows  a search  pattern  which  searchs  for  the  target  over  an  area  which 
is  nine  times  the  area  of  the  original  search  area  in  eight  discrete 
steps  of  the  seeker  LOS.  Figure  32(b)  shows  a search  pattern  which 
searchs  for  the  target  over  an  area  which  is  four  times  that  of  the 
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original  area  In  four  discrete  steps.  Search  pattern  32(b)  would 
require  less  than  half  the  time  as  search  pattern  32(a)  if  say  the 
target  Is  in  the  last  possible  dotted  square.  Due  to  recent  experiments 
conducted  at  HI COM  where  the  missile  seeker  was  slaved  to  the  PTS^ 

It  is  felt  that  the  search  pattern  of  Figure  32(b)  will  be  adequate 
(i.e.,  the  probability  of  finding  the  target  in  search  pattern  32(b) 
is  quite  high).  However,  in  a technology  program,  tests  should  be 
conducted  to  determine  the  optimal  search  pattern  (that  is  one  that 
yields  a high  enough  probability  of  finding  the  target  in  the  least 
possible  time).  Another  reason  for  choosing  search  pattern  32(b)  is 
because  of  the  inaccuracies  of  the  correlation  algorithm  around  the 
edges  of  the  FOV.  Since  the  edges  of  the  original  FOV  in  Figure  32(b) 
are  well  within  the  search  pattern  FOVs,  the  possibility  of  missing 
the  target  due  to  edge  effect  is  reduced.  If  one  uses  the  search 
pattern  of  Figure  32(a),  there  exists  the  possibility  of  not  finding 
the  target  at  all  even  though  it  is  within  the  original  FOV  but  close 
to  an  edge. 

The  correlation  values  are  ignored  during  the  slewing  of  the  seeker 
LOS.  When  the  LOS  stops,  if  after  n TV  fields  the  correlation  does 
not  exceed  the  chosen  threshold,  the  LOS  is  advanced  to  the  next  search 
area.  If  after  completely  searching  all  areas  the  threshold  is  not 
exceeded,  the  gunner  would  be  flashed  a s^jnal  to  this  effect  whereupon 
he  can  re-initiate  the  process  if  he  desires. 


4.  IMPLEMENTATION 


The  system  required  to  implement  either  of  the  correlation  and 
location  functions  presented  in  Chapter  3 will  consist  primarily  of 
hardware  to  perform  the  correlation  operation  itself.  The  speed  of 
operation  of  the  entire  system  w.ll  be  limited  by  speed  considerations 
in  the  correlator.  The  correlation  activity  consists  of  a comparison 
activity  (absolute  difference,  equivalence  ((^)  or  multiplication) 
followed  by  a summation  operation.  In  order  to  speed  the  summation 
process,  an  adder  tree  is  employed.  The  number  of  layers  of  addition 
and  hence  the  speed  of  the  adder  tree  is  a function  of  the  fan-in  or 
number  of  branches  on  each  layer  of  the  tree  as  well  as  the  propagation 
delay  of  the  logic  used  to  implement  the  tree.  It  is  of  major  importance 
that  maximum  fan-in  be  employed  whenever  possible.  Digital  implemen- 
tations treat  addition  as  a binary  operatior  . Therefore,  the  number  or 
layers  in  the  adder  tree  is  directly  related  to  log2N  where  N is  the 
numoer  of  functions  to  be  summed.  Dramatic  speed  Ircreases  will  be 
obtained  by  increasing  the  fan-in.  A later  section  will  illustrate 
this  phenomenon.  The  method  of  presentation  will  be  to  discuss  the 
various  components  of  Figure  33  by  first  examining  correlation  techniques 
followed  by  discussion  of  the  various  support  components. 

A.  Implementation  of  One-Bit  by  One-Bit  Real-Time  Correlator 

In  this  section  the  logic  design  for  a one-bit  by  one-bit  corre- 
lator is  considered.  The  objective  is  to  find  the  largest  reference 
array  which  can  be  handled  in  real  time  subject  to  a variety  of  con- 
straints such  as  logic  family,  power  requirements,  etc.  It  has  been 
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Block  diagram  of  correlator  hanc'-off  system. 
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shown  that  each  value  in  the  R correlation  matrix  must  be  found  In  about 
133  nsec  when  sampling  the  Incoming  video  at  7.5  MHz.  Two  logic  families 
will  be  Investigated:  ECL  and  T7L  Schottky.  No  slower  logic  families 
can  meet  the  computational  speed  requirement  for  even  the  smallest 
reference  array.  The  technique  employed  Is  to  use  an  exclusIve-OR  or  an 
exclusIve-NOR  circuit  to  perform  difference  or  multiplication  respectively 
and  then  to  sum  these  correlation  elements  with  an  adder  tree. 

Figure  34  is  the  logic  diagram  for  a 12  by  1 correlator.  One  can 
think  of  this  one  column  in  an  array  correlator.  The  inputs  to  the 
exclusive-OR  gates  are  quantized  one-bit  signals.  Figure  35  shows  how 
16  of  these  correlators  can  be  combined  to  yield  a 12  x 16  element 
correlator.  The  circuit  type,  packaging,  power,  and  propagation  times 
are  given  in  the  Figures  below  the  appropriate  blocks.  All  M x L 
correlators  can  be  designed  in  the  same  way.  The  12  x 16  correlator 
was  chosen  for  illustrational  purpose?. 

Table  4 is  a summary  of  total  packaging  and  power  requirements, 
and  total  propagation  delay  for  several  reference  array  sizes  using  ECL 
and  TTL  logic.  The  largest  real-time  correlator  us^ng  ECL  logic  is 
seen  to  be  48  x 64.  The  largest  array  using  TTL  logic  is  less  than 
12  X 16  aid  is  not  shown  since  this  is  unacceptable.  One  can  observe 
from  Table  4 that  the  ECL  logic  is  about  twice  as  fast  as  the  TTL  logic 
for  a given  array  size.  However,  the  total  power  requirement  is  about 
twice  as  great  for  the  ECL  logic  as  for  the  TTL  logic.  Since  the 
number  of  packages  is  about  the  same,  the  physical  size  of  the  corre- 
lators should  be  about  the  same  unless  heat  sinks  are  necessary  for  the 


ECL  correlator. 
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Figure  Logic  for  12  x 1 correlator 


106 


ECL 

TTL 

Array  Size 

Pkg 

Power 

Delay 

Pkg 

Power 

Delay 

8 X 12 

82 

29  W 

51  ns 

74 

15,  W 

114  ns 

12  X 16 

166 

59  W 

64  ns 

150 

30  M 

139  ns 

16  X 24 

366 

120  W 

77  ns 

304 

61  W 

164  ns 

24  X 32 

671 

239  W 

90.5  ns 

611 

123  W 

189  ns 

32  X 48 

1345 

480  W 

108.5  ns 

48  X 64 

2696 

964  W 

126.5  ns 

Table  4.  Comparison  of  ECL  vs.  TTL  logic  correlators. 


It  is  possible  to  extend  the  hardware  in  a simple  way  to  encompass 
larger  arrays  with  little  increase  in  hardware  complexity.  The  basic 
design  of  all  the  fast  correlators  is  a multiply  circuit  and  an  adder 
tree.  In  the  one-bit  case  it  is  possible  to  "hang"  additional  multiply 
circuits  on  the  adder  tree  without  effecting  the  propagation  times 
considerably.  The  full  adder  is  in  fact  a ternary  function.  Figure  36 
shows  the  logic  diagrams  for  a 15  x 1 correlator.  Figure  3/  is  the 
exteiision  of  Figure  3d  that  produces  a 15  x 16  correlator.  This  basic 
block  is  iterated  in  space  to  form  the  various  array  sizes  listed  in 
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Figure  36.  Logic  diagram  for  a 15  x 1 one-bit  correlator. 
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15  X 16 
Correlator 


1/4  ckt 

1/4  ckt 

1/2  ckt 

Output 

MC10181 

MCI  0181,  MCI  0180 

MCI  0181 

MC10181 

8 pkg 

6 pkg 

4 pkg 

2 pkg 

4800  NW 

3120  MW 

2400  NW 

1200  NW 

6.5  NS 

11  NS 

13  NS 

13  NS 

Figure  37.  Extension  of  Figure  36  to  produce  a 
15  X 16  correlator. 


Array  Size 

Pkg. 

Power 

— 

Delay 

15  X 16 

166 

59w 

64NS 

16  X 30 

366 

120w 

77MS 

30  X 32 

671 

239w 

90.5NS 

32  X 60 

1345 

480w 

108. 5NS 

60  X 64 

2696 

964w 

126. 5NS 

Table  5.  One^bit  by  one-bit  correlator  using 
ECL  logic. 


It  appears  unlikely  that  a complete  ECL  Implementation  would  be 
practical  due  to  the  large  power  requirement.  In  this  sev:t1on  we  will 
first  describe  an  implementation  based  on  low  power  Schottky  TTL-logic. 
Since  this  logic  family  is  not  fast  enough  to  do  an  addition  of  2048 
(32  X 64)  one-bit  numbers  in  anything  apnrjaching  133  ns,  it  is 
necessary  to  perform  the  computation  In  pieces.  This  technique  is  called 
pipelining.  During  a giver  time  interval  the  'pipe'  may  be  computing 
Part  1 of  Item  3,  Part  2 of  Item  2 and  Part  3 of  Item  1;  at  the  next 
time  step  computation  is  performed  on  Part  1 of  Item  4,  Part  2 of  Item 
3,  Part  3 of  Item  2.  A new  result  is  generated  at  each  time  interval  but 
for  any  specific  data  item  the  time  required  is  the  product  of  the  basic 
time  interval  and  the  number  of  sections  of  pipe  (see  Figure  38).  The 
Schottky  pipe  version  illustrated  on  the  accompanying  pages  (Figure  39) 
requires  approximately  1600  packages  and  300  Watts  for  a 32  x 64  array. 
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Figure  38.  Items  In  pipe. 


The  pipe  delay  In  each  of  the  three  sections  of  pipe  is  132  ns. 
Memory  required  in  addition  to  that  contained  In  the  correlator  itself 
Is  assumed  to  be  a charged  coupled  device  since  this  will  provide  the 
necessary  speed  with  very  low  power  consumption.  Bubble  memories  may 
become  practical  in  this  application  as  well.  Assuming  a picture  image 
P X Q with  R points  devoted  to  horizontal  retrace  and  sync,  and  a corre- 
lation array  K x L,  from  Figure  40  we  observe  that  the  delay  required  is 
(Q  + R - L)  for  each  row.  Using  CCD  (charge  coupled  array)  power  dissipa 
tion  at  10  MHz  is  about  20  yW/BIT.  Therefore  total  power  dissipation  is 
20  X 10"^K(Q  + R - L)  milliwatts. 
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Fiqure  (Part  1).  Correlator  and  adder  tree. 
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Figure  39  (Part  4).  Correlator  and  adder  tree. 


(Q  + R - L)  Bits  each  row 
K(Q  + R - L)  Bits  total 

Figure  40.  Memory  requirement  for  correlation  computation. 

Another  problem  involving  the  one-bit  by  one-bit  correlator  concerns 
the  removal  of  the  OC  component  of  the  LR  image.  This  forms  a support 
component  in  the  system  block  design.  The  following  is  a discussion  of 
a sub-optimal  technique  that  accomplishes  the  removal  of  the  DC  digitally. 
Referring  to  Figure  41,  in  approximating  the  level  about  which  to  quantize 
pixel  A the  value  of  pixel  B must  be  known.  Pixel  B lags  A by  (K/2)(Q+R)  + 
I '2  sample  Intervals,  The  column  containing  pixel  B is  used  to  determine 


Figure  41.  Method  of  computing  mean  video  level. 
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L mm  values  and  then  Is  not  used  until  Q -t-  R - L events  later.  Data 
Item  C Must  be  removed  from  the  column  sum  after  K(Q  * R)  events.  The 
resulting  algorithm  is  presented  as  Figure  42.  If  we  assume  a 3 bit 
quantization  of  the  picture,  then  3 x [K  x (Q  + R)  + L/2]  + 14  x (Q  + R + L) 
bits  of  memory  are  required  As  before,  using  Q R 476,  K » 32,  L > 64, 
53,352  bits  are  required.  Power  dissipation  is  approximately  1.5  watts 
for  the  memory  array  using  CCD.  Clocking  and  Interface  logic  could  increase 
this  number  significantly. 

Mean  Value 

(Shift  to  divide 


3x  (K(Q  + R)  + L/2) 

+ I4x  (q  + R + L)  Bits  of  memory 


Figure  42.  Mean  video  computation  and  quantizer. 
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The  use  of  half-frane  correlation  (I.e.,  non- Interlaced  Images) 
serves  to  reduce  the  hardware  memory  'requirements.  In  order  to  achieve 
reasonable  computational  speeds  through  the  adder  tree,  the  circuit  Is 
divided  Into  sections  (I.e.,  pipelined).  With  Interlacing,  one  of  the 
final  stages  of  the  pipe  sums  results  delayed  from  the  previous  half- 
frame. Because  this  delay  requires  considerable  memory  and  because 
of  the  effects  of  vibration  discussed  earlier.  It  is  reconmended  that 
a_  ha'] f-f mjg_^rTe]jUg^  be  1mplemente^^hu£^^^effectlM  space 

The  maximizing  circuit  Is  quite  simple  and  Is  shown  In  Figure  43  . 
The  position  counter  Is  also  used  In  the  edge  correction  problem.  The 
picture  Is  dimensioned  as  in  Figure  44-  . notice  that  both  the  hori- 
zontal and  vertical  blanking  intervals  are  indicated  by  a one  in  the 
most  significant  bit  of  each  counter.  The  retrace  intervals  have  been 
adjusted  to  ensure  this  simple  hardware  implementation.  The  final 
construction  may  require  that  sample  rates  be  adjusted  to  achieve  this 
effect. 

The  mean  value  of  the  image  needed  for  quantization  near  the  edges 
is  difficult  to  calculate  because  part  of  the  K X L subarray  extends 
into  the  horizontal  retrace  region  or  vertical  retrace  region  as  shown 
in  Figure  45.  The  dc  component  of  these  areas  that  depend  on  values 
outside  the  field  of  view  can  be  calculated  by  substituting  a binary 
noise  source  for  the  retrace  voltage.  One  possible  implementation  is 
shown  in  Figure  46.  This  will  preclude  the  possibility  of  false  con- 
trast changes  near  the  edge  by  smearing  the  edge  so  that  it  always 


appears  gray. 


fflln/max 


1 

P X Q is  single  scan  picture  area 

For  example  256  x 512 

R and  S are  horizontal  and  vertical 
blanking  intervals 

For  example  (P  + S)  x (Q  + R)  could 
be  320  X 576 


Figure  44,  Picture  size  (P  x Q)  with  horizontal  and 
vertical  retrace  R and  S. 


no  picture  in  these  regions 


Figure  45.  Retrace  ^Edge)  effects  on  quantization. 
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Figure  46.  Binary  noise  source  generator. 


The  high  resolution  image  contains  far  too  much  data  to  be  used  in 
the  correlation.  This  data  is  reduced  by  coi^utlng  a spatial  average 
using  Equation  67  where  and  Uy  are  assumed  to  be  integers.  The  com> 
ponents  of  the  spatial  averaging  are  then  sampled  to  reduce  the  data. 

Figure  47  illustrates  the  hardware  for  such  a reduction.  The  digital 
approach  uses  MOS  or  CCD  shift  registers  for  delay.  The  precision  of  the 
HR  image  is  assumed  to  be  3-bits.  A binary  adder  with  a register  acting 
as  accumulator  forms  the  desired  sums.  The  sum  of  25  3-bit  uui-nbers  will 
not  exceed  256  or  8 bits  in  precision.  The  Divider  is  a 2048  x 1 ROM. 

3x4,  3x5,  4x3,  4x4,  4x5,  5x3,  5x4,  5x5  horizontal  and  vertical  reductions 
in  the  aspect  ratios  can  be  handled.  The  RON,  also  causes  the  3-bit 
precision  of  the  image  to  be  quantized  to  1-bit. 

The  horizontal  and  vertical  blanking  signals  are  used  to  force  the 
magnitude  comparator  In  the  peak  detecting  circuit  to  ignore  the  corre- 
lation values  computed  during  horizontal  and  vertical  retrace.  Since 
the  correlator  is  designed  to  run  free,  these  correlation  values  must  be 
ignored  during  retrace.  The  circuit  used  to  accomplish  this  Is  shown  in 
Figure  48.  Also  Illustrated  is  a method  for  selecting  either  minimum 
or  maximum.  If  B is  to  be  larger  than  A to  track  maximum*  then  the 
blanking  should  force  B > A. 

B.  Block  Diagram  of  32  x 64  Array  Correlator  System 
A detailed  block  diagram  of  a total  correlator  system  has  been  drawn 
by  incorporating  all  of  the  block  diagrams  of  the  previous  figures.  The  cor- 
relation system  i«  designed  around  the  XOR  implementation.  The  block  diagram 


5 Unit 
Delay 


Figure  47,  Averaging  circuit  for  resolution  reduction  of  HR  video. 
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Figure  48.  Blanking  circuits  for  correlator  during 
horizontal  and  vertical  retrace. 


also  shows  the  pre-processing  that  Is  done  on  both  the  HR  and  LR  video 

signals  assuming  an  adJusUble  FOV  ratio  between  the  two  and  also  shows 

how  the  processed  signals  are  clocked  Into  the  correlator. 

The  completed  Implementation  Is  shown  In  Figure  33.  A more  detailed 

representation  Is  shown  in  fold  out  Figure  49.  It  Is  Important  to  remember 

that  the  diagrams  represent  potential  ways  of  Implementing  the  desired 

design  and  are  presented  as  a feasibility  demonstration.  A practical 

Implementation  might  not  employ  Identical  techniques.  The  following 

discussion  treats  the  blocks  in  Figui'e  33  as  a system.  The  component 

functions  were  treated  separately  earlier  In  this  chapter.  The  high 
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resolution  signal  is  separated  into  video  and  sync  components.  The 

vertical  sync  is  used  ^s  a reference  to  a phase-locked  loop  circuit  (PLL) 

to  provide  the  HR  clock  which  is  used  to  control  the  HR  sample  interval. 

The  HR  video  is  conditioned  by  a h^gh  pass  filter  to  remove  the  mean  or 

dc  component  of  that  signal.  The  HR  sigr<a1  is  then  spatially  averaged 

in  a field  of  varying  size  to  reduce  resolution  to  that  compatible 

with  the  LR  image.  Selected  pulses  from  the  HR  clock  (again  to  reduce 

resolution)  are  used  to  load  the  HR  portion  of  the  correlator.  The  HR 

buffer  is  not  loaded  on  every  HR  frame  but  at  a slower  rate.  This  rate 

may  be  target  dependent.  The  LR  sync  is  used  as  a reference  PLL  circuit 

in  a manner  similar  to  the  HR  sync.  DC  removal  is  accomplished  on  a 

pixel  basis  by  using  an  approximate  local  averaging  technique.  This 

processed  video  is  sent  to  the  LR  correlator  buffer.  The  correlator 

produces  correlation  signals  at  the  LR  clock  rate.  The  peak  of  this 

signal  is  passed  to  a peak  averaging  circuit  that  reduces  the  probability 

of  false  registration.  The  y coordinate  is  adjusted  on  odd  frames.  If 

the  maximum  detector  fails  to  find  a maximum  above  the  specified  threshold 

in  a specified  number  of  consecutive  fields,  the  search  mode  signal  is 
activated  in  an  attempt  to  locate  the  target.  The  correlator  speed  is 

the  limiting  factor  in  this  implementation. 

The  speed  of  the  correlator  is  limited  by  the  fan-in  at  each  step 
of  the  adder  tree.  The  investigators  have  recently  learned  of  an 
integrated  circuit  available  from  TRW  that  forms  the  XOR  of  64  pairs  of 
numbers  and  sums  the  resulting  products  by  analog  means.  (The  summing 
is  done  by  adding  currents  at  a node.)  The  speed  of  this  device  is  quoted 
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In  the  5-10  MHz  range.  It  may  be  possible  to  enq?1oy  this  device  in  a 
correlator  implementation  that  does  not  require  pipeline  techniques.  The 
power  requirements  for  this  device  are  also  well  below  the  full  digital 
implementation.  The  analog  adder  is  less  precise  than  the  digital  one 
but  the  system  should  perform  well  in  either  approach.  The  investigators 
recommend  the  analog  approach  in  view  of  present  available  information. 
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5.  CONCLUSIONS  AND  RECOMMENDATIONS 

Most  of  the  conclusions  and  recomnendatlons  resulting  from  the 
work  on  this  contract  have  been  given  and  justified  within  the  first 
four  chapters  of  this  report.  For  compactness  they  are  given  again  In 
this  chapter. 

A.  Conclusions 

A one-bit  by  one-bit  correlator  using  exclusive  OR  gate  or  exclusive 
NOR  gate  realization  should  be  used  In  a hand-off  technology  program. 

2.  The  one-bit  correlator  may  be  either  an  implementation  of  the 
Direct  Sunmatlon  Method  (search  for  a maximum)  or  the  Sequential 
Simularity  Detection  Algorithm  (search  for  a minimum)  since  they 
are  equivalent  in  a one-bit  by  one-bit  correlator. 

3.  In  a technology  program  the  reference  array  size  should  be  /ariable 
in  order  to  investigate  the  trade-off  of  reference  array  size  vs. 
correlation  accuracy  using  typical  military  targets  and  scenes. 

The  size  of  the  reference  array  in  the  correlator  should  be  variable 
up  to  at  least  32  x 64  or  2048  elements.  The  statistical  reliability 
of  the  correlator  improves  as  the  number  of  elements  increases. 

4.  The  adder  tree  may  be  either  digital  or  analog.  The  digital  adder 
is  more  accurate  but  requires  more  hardware  than  the  analog  imple- 
mentation. There  exists  more  clock  timing  problems  with  the 
digital  adder  tree. 
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5.  The  exclusive  OR  gates  In  the  ccrrelator  should  be  in  special 
purpose  chips*  like  the  one  developed  by  TRW  and  reported  in 
Chapter  Four  of  this  report.  These  special  chips  reduce  the 
amount  of  hardware  significantly.  Other  companies  may  manufacture 
similar  chips  but  they  are  unknown  to  these  investigators. 

6.  The  32  X 64  correlator  should  operate  on  32  succeeding  lines  in 
each  TV  field  rather  than  16  lines  from  one  field  and  the  16 
interlaced  lines  from  the  next  TV  field.  This  implementation  Is 
referred  to  as  a half-frame  correlator  and  reduces  the  required 
hardware  and  minimizes  the  effects  of  vibration. 

7.  Some  form  of  image  normalization  is  required  to  prevent  false 
registration  points.  For  the  one-bit  by  one-bit  correlator  no 
normalization  is  required  since  the  images  are  already  normalized. 

8.  If  a higher  level  quantizer  is  desired,  there  is  not  much  point 
in  using  more  than  four  levels  from  a S/N  point  of  view.  If  more 
than  two  levels  are  used,  care  should  be  taken  in  how  the  quantiza- 
tion step  levels  are  obtained.  Inaccuracies  in  the  quantization 
levels  between  the  two  video  signals  could  cause  a larger  reduction 
in  the  S/N  ratio  than  that  gained  by  going  to  more  levels.  Using 
more  than  two  levels  can  cause  further  degradation  because  fewer 
picture  elements  can  be  processed  in  a given  time. 

9.  At  the  ranges  at  which  the  correlator  will  be  operating,  It  is 
actually  correlating  on  an  area  and  not  the  target  since  the 
target  represents  only  a small  portion  of  the  reference  image. 
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10.  With  a 32  X 64  or  2048  pixel  reference  array,  the  performance  of 
the  correlator  will  depend  almost  entirely  on  the  goodness  of  the 
preprocessing  of  the  two  Images.  The  various  preprocessing  design 
parameters  should  be  investigated  in  a technology  program.  Realistic 
sensor  differences  should  be  included  in  this  investigation. 

11.  The  search  pattern  shown  in  Figure  32(b)  Is  recommended  If  the 
target  is  not  found  In  the  Initial  missile  seeker  FOV. 

B.  Objectives  of  a Technology  Program 

From  the  results  of  this  study  it  Is  concluded  that  hand-off  of  a 
target  from  a pointing  and  tracking  system  to  a missile  seeker  using  a 
correlator  is  not  only  feasible  but  an  attractive  option.  It  is 
recommended  that  a correlator  be  built  and  used  in  a technology  program 
to  demonstrate  this  feasibility  and  to  study  the  various  trade-offs 
discussed  in  this  report.  The  following  are  some  recomnended  objectives 
of  such  a technology  program. 

1.  Determine  overall  performance  of  correlator.  Performance 
criteria  include  average  time  for  target  lock-on,  percentage 
of  false  lock-ons,  percentage  of  missed  targets,  etc. 

2.  Investigate  the  accuracy  of  the  correlator  vs.  the  size  of 
reference  array  used.  This  might  be  scene  dependent. 

3.  Determine  the  optimal  threshold  value  to  use  in  the  correlator. 
This  is  also  scene  dependent  to  some  extent. 

4.  Determine  the  effects  of  varying  the  TV  signal  sampling  rate 
from  approximately  3.5  MHz  to  8 MHz. 
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5.  Investigate  sensitivity  of  correlation  algorithm  to  quantization 
about  Incorrect  dc  level . 

6.  Investigate  sensitivity  of  correlation  algorithm  to  errors  In 

aid  In  reducing  the  resolution  of  the  PTS  video  to 
that  of  the  seeker  video. 

7.  Investigate  effects  of  vibration  on  the  correlator  accuracy. 

8.  Determine  sensitivity  of  correlator  to  sensors  with  differing 
spectral  responses. 

9.  Determine  rate  at  which  HR  reference  Image  must  be  up-dated 
with  typical  moving  targets. 

10.  Investigate  sensitivity  of  correlator  to  various  video  S/N 
ratios. 
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